-- Logs begin at Tue 2022-08-30 14:01:20 UTC, end at Tue 2022-08-30 15:16:01 UTC. -- Aug 30 14:36:53.168632 dv-cinder-828807-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 30 14:36:53.178085 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 30 14:36:53.180010 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 30 14:36:53 2022] *** Aug 30 14:36:53.180010 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 30 14:36:53.180010 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 30 14:36:53.180010 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: nodename: dv-cinder-828807-5-iscsi Aug 30 14:36:53.180010 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: machine: x86_64 Aug 30 14:36:53.180010 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: clock source: unix Aug 30 14:36:53.180010 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: pcre jit disabled Aug 30 14:36:53.180010 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: detected number of CPU cores: 4 Aug 30 14:36:53.180010 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: current working directory: / Aug 30 14:36:53.180010 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: detected binary path: /usr/bin/uwsgi-core Aug 30 14:36:53.180010 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: your processes number limit is 47680 Aug 30 14:36:53.180623 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: your memory page size is 4096 bytes Aug 30 14:36:53.180623 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: detected max file descriptor number: 2048 Aug 30 14:36:53.180623 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: lock engine: pthread robust mutexes Aug 30 14:36:53.180623 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: thunder lock: enabled Aug 30 14:36:53.180623 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 30 14:36:53.180623 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 30 14:36:53.219415 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: Python main interpreter initialized at 0x563c213859c0 Aug 30 14:36:53.219415 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: python threads support enabled Aug 30 14:36:53.219415 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: your server socket listen backlog is limited to 100 connections Aug 30 14:36:53.219415 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: your mercy for graceful operations on workers is 90 seconds Aug 30 14:36:53.219733 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: mapped 403077 bytes (393 KB) for 2 cores Aug 30 14:36:53.219829 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: *** Operational MODE: preforking *** Aug 30 14:36:53.220097 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: *** uWSGI is running in multiple interpreter mode *** Aug 30 14:36:53.220097 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: spawned uWSGI master process (pid: 88046) Aug 30 14:36:53.220283 dv-cinder-828807-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 30 14:36:53.221342 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: spawned uWSGI worker 1 (pid: 88058, cores: 1) Aug 30 14:36:53.221342 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: spawned uWSGI worker 2 (pid: 88059, cores: 1) Aug 30 14:36:53.221342 dv-cinder-828807-5-iscsi devstack@keystone.service[88046]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 30 14:36:57.553139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 14:36:57.553139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: from cryptography.utils import int_from_bytes Aug 30 14:36:57.553884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 14:36:57.553884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: from cryptography.utils import int_from_bytes Aug 30 14:36:57.554299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 14:36:57.554299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: from cryptography.utils import int_from_bytes Aug 30 14:36:57.554428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 14:36:57.554428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: from cryptography.utils import int_from_bytes Aug 30 14:36:57.603752 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 14:36:57.603950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 14:36:57.604160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] command line args: [] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 14:36:57.604417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 14:36:57.604602 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ================================================================================ {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 14:36:57.604859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] admin_token = **** {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.605106 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] config_dir = [] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.605296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.605479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] config_source = [] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.605660 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] debug = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.605861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.606221 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.606402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.606655 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.606905 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.607159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] list_limit = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.607312 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 14:36:57.607399 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] log_config_append = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.607512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 14:36:57.607625 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.607726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] log_dir = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.607859 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] command line args: [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 14:36:57.608000 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] log_file = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.608179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 14:36:57.608322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.608434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ================================================================================ {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 14:36:57.608554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.608673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] admin_token = **** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.608792 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.608904 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] config_dir = [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.608904 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.609118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.609233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] config_source = [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.609349 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.609349 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.609580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] debug = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.609580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.609826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.609950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.610061 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.610174 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.610296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.610399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.610507 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.610625 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.610742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.610858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] list_limit = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.610974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.611084 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] log_config_append = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.611225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.611348 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.611348 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] log_dir = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.611569 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.611681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] log_file = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.611797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.611915 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.611915 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.612125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.612261 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.612363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] publish_errors = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.612467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.612595 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.612707 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.612824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.612936 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.613061 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.613184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.613304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.613415 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.613526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.613636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.613758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] standard_threads = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.613870 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.613980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.614096 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.614206 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.614323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.614486 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.614486 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] use_journal = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.614709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.614820 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] use_json = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.614950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.615073 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] use_stderr = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.615073 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] use_syslog = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.615324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.615438 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.615438 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.615651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.615651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] publish_errors = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.615870 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.615986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.616098 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.616219 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.616331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.616443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.616553 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.616663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.616780 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.616899 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.617011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.617120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] standard_threads = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.617237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.617349 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.617460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.617645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.617760 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.617877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.617877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] use_journal = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.618091 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.618202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] use_json = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.618324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.618437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] use_stderr = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.618437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] use_syslog = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.618645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.618762 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:36:57.618879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.618999 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.619131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.619246 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.619385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.619385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.627265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.627265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.627461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.627579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.627654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.627722 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.627802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.627802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.627938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.627938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.628061 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.628127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.628249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.628329 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.628404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.628473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.628540 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.628613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.628692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.628762 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.628830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.628897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.628970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.628970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.629100 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.629169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.629243 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.629312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.629379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.629446 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.629585 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.629656 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.629725 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.629797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.629871 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.629945 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.630020 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.630090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.630158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.630225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.630298 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.630366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.630441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.630510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.630577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.630650 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.630719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.630786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.630864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.630933 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.631000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.631067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.631167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.631269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.631375 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.631470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.631567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.631666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.631760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.631858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.631960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.632055 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.632148 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.632241 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.632349 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.632431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.632502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.632616 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.632689 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.632801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.632874 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.632997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.633071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.633141 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.633255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.633328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.633441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.633558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.633681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.633757 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.633880 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.633955 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.634068 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.634147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.634217 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.634338 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.634412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.634525 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.634603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.634674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.634794 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.634906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.635023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.635118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.635239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.635314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.635427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.635513 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.635584 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.635696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.635770 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.635881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.635955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.636075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.636149 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.636224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.636345 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.636345 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.636520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.636601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.636671 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.636803 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.636880 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.636948 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.637060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.637145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.637223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.637292 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.637359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.637426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.637509 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.637632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.637707 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.637776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.637844 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.637912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.637979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.638116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.638188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.638304 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.638377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.638446 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.638514 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.638590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.638660 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.638660 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.638788 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.638856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.638923 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.638923 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.639068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.639166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.639246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.639316 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.639390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.639459 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.639534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.639604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.639718 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.639792 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.640360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.640431 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.640529 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.640649 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.640708 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.640708 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.640831 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.640899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.640978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.641049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.641117 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.641185 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.641251 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.641324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.641400 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.641470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.641544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.641614 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.641681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.641749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.641823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.641893 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.641961 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.642028 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.642101 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.642169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.642244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.642360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.642434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.642502 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.642570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.642638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] auth.external = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.642714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.642784 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.642852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.642852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.642975 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.643043 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.643135 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.643264 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.643339 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] auth.password = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.643408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.643476 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] auth.token = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.643543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.643642 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.643723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.643801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.643871 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.643941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.644016 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.644087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.644155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.644234 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.644311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.644382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.644382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.644505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.644574 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] credential.caching = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.644653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.644725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.644793 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] auth.external = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.644793 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.644929 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.644929 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.645079 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.645150 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.645219 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.645293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.645367 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] auth.password = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.645436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.645512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] auth.token = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.645582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.645650 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.645650 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.645774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.645774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.645913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.645913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.646038 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.646114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.646184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.646260 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.646330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.646399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.646467 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.646535 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] credential.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.646603 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.646682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.646752 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.646866 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.646948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] federation.caching = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.647134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.647216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.647389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.647472 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.647638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.647723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.647872 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.647962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.648120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.648199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.648357 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.648434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.648601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.648678 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.648842 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.648921 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.649099 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.649176 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.649340 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.649418 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.649593 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.649671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.649832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity.caching = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.649910 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.650081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.650158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.650317 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] federation.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.650394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.650555 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.650638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.650793 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.650871 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.651039 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.651135 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.651295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.651372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.651537 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.651615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.651773 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.651902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.652028 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.652161 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.652270 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.652397 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.652526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.652631 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.652767 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.652866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.653018 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.653116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.653258 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.653365 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.653506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.653613 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.653755 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.653871 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.653994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.654122 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.654237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.654398 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.654475 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.654639 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.654978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.655186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.655270 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.655466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.656156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.656377 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.656615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.656851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.657085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.657331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.657580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.657890 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.658122 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.658373 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.658491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.658645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.658763 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.658884 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.658999 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.659131 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.659247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.659429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.659547 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.659672 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.659787 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.659900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.660012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.660125 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.660238 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.660358 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.660479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.660590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.660701 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.660812 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.660948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.661071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.661252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.661369 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.661480 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.661599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.661711 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.661829 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.661942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.662052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.662169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.662281 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.662413 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.662536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.662650 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.662761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.662872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.663004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.663176 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.663305 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.663419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.663530 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.663643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.663754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.663865 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.663984 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.664097 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.664208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.664318 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.664436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.664549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.664692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.664810 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.664922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.665033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.665144 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.665285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.665394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.665491 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.665585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.665680 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.665831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.665950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.666085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.666189 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.666318 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.666429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.666556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.666658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.666807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.666911 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.667039 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.667165 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.667310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.667413 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.669505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.669614 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.669693 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.669758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.669828 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.669897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.669975 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.670046 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.670116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.670185 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.670253 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.670322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.670398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.670469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.670538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.670606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.670674 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.670743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.670819 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.670890 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.670959 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.671028 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.671135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.671222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.671295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.671364 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.671434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.671503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.671572 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.671654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.671727 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.671796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.671864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.671933 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.672002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.672079 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.672150 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.672219 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.672288 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.672357 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.672426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.672571 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.672645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.672714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.672783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.672852 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.672927 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.673004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.673075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.673246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.673326 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.673491 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.673567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.673733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.673811 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.673968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.674085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.674206 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.674340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.674452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] resource.caching = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.674580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.674705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.674822 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.674988 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.675144 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] resource.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.675325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.675508 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.675654 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.675770 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.675883 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.676005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.676120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.676234 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.676347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.676498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.676613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.676734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.676848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.676961 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] role.caching = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.677074 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.677186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] role.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.677301 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.677421 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.677542 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] role.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.677656 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.677768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] role.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.677881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.677995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.678116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.678231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.678346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.678460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.678574 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.678687 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.678808 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.678922 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.679035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.679173 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.679288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.679400 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.679524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.679638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.679749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.679860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.679972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.680083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.680206 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.680319 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.680431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.680542 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.680726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.680850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.680995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.681110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.681354 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.681475 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.681589 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.681710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.681825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.681939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.682051 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.682164 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.682305 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.682429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.682545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.682657 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.682770 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.682882 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.683016 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.683174 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.683295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.683408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.683522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.683636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.683749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.683870 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.683985 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.684098 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.684211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.684322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.684436 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.684556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.684673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.684784 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.684898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.685010 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.685123 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.685244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.685358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] token.caching = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.685471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.685584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.685697 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.685810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.685931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] token.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.686046 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.686214 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.686326 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.686421 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.686516 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.686636 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.686771 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.686886 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.686999 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.687141 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.687258 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.687384 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.687500 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.687613 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.687728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.687841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.687954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.688074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.688188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.688301 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.688414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.688526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.688647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.688761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.688875 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.688987 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.689109 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.689231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.689346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.689468 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.689590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.689712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.connection = **** {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.689829 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.689942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.690055 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.connection = **** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.690168 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.690280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.690401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.690515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.690629 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.690742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.690854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.690967 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.691171 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.691290 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.691403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.691516 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.691629 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.691742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.691862 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.691978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.692092 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.692205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.692317 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.692430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.692550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.692664 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.692777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.692890 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.693004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.693117 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.693288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.693407 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.693520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.693632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.693744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.693883 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.693988 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.694087 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.694182 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 14:36:57.694277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:36:57.694372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88059) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 14:36:57.724258 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.724484 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.724574 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.724718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.724802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.724950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.733009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.733504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.760780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.761002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.761352 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.761587 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.816759 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.821725 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.866114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.866339 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.866753 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:57.866991 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.020319 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.020556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.020785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.021008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.021234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.021454 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.022769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.023003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.023257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.023482 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.023700 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.023918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.045656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.045807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.045907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.046057 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.046138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.046361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.046444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.046544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.046638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.046758 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.046838 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.046957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.047050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.047194 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.057796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.058204 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.072554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.072659 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.072748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.072823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.072953 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.073036 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.073177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.073259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.073396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.073478 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.073609 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.073691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.073823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.073913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88059) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:36:58.102241 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.102391 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.102476 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 14:36:58.102602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 14:36:58.102776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.102933 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.103014 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 14:36:58.103159 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 14:36:58.103269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.103438 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.103531 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.103714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.103973 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.104072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.104233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.104327 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.113746 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.113844 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.114027 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.114107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.114298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.124810 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.124994 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.125071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.125143 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.125215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.125299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.125375 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.125446 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.125517 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.125595 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.125675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.125812 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.125894 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.126097 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.126178 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.126364 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.126453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.126639 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.126719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.126917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.126996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.127215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.127329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.127496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.127610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.127766 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.127886 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.150988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.151348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.152489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.152771 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.152894 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.153099 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.157644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.157929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.158122 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.158203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.158409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.158691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.164715 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.165002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.165083 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.170060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.170143 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.170143 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.170271 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.170337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.170404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.170472 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.170472 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.170580 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.170580 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.170741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.171032 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.171351 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.171640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.172095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.172389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.172694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.188821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.189099 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.189538 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.189822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.192990 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.193708 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.196119 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.198737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.198826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.198826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.200768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.201649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.203962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.204908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.207048 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.207373 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.207661 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.207949 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.208199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.208281 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.208503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.208598 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.208793 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.209081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.209372 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.209650 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.224926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.225216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.225488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.225758 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.226177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.226470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.226738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.227003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.232538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.234607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.234782 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.236197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.236355 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.236608 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.237744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.238023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.239280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.239571 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.239855 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.240145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.240444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.240743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.241032 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.241432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.241723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.242016 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.251760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.252047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.252334 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.252620 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.252905 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.253182 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.253460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.253539 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.253777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.254156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.254442 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.254727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.255010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.255326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.268904 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.270788 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.272104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.273864 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.274986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.276467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.276760 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.277946 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.278196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.278278 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.278503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.280122 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.280405 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.281210 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.285866 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.287364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.287749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.289227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.290237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.290532 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.290819 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.291128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.292279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.292575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.292865 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.293154 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.299659 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.299950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.300231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.301755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.302051 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.302337 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.307160 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88059) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.307443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.307724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.308001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.320593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.320593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.320593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.320593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.320593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.320989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:36:58.321073 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.321148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.321220 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.321292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.321364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.321444 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.321518 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.321590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.321662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88059) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.321734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.321734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.321734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.321734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.321734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.321734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.322123 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.322123 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.322123 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:36:58.345644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563c213859c0 pid: 88059 (default app) Aug 30 14:36:58.348344 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563c213859c0 pid: 88058 (default app) Aug 30 14:36:58.349053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63d094-5510-48c9-97e4-d95063a30429 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:58.349343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63d094-5510-48c9-97e4-d95063a30429 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:58.349620 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63d094-5510-48c9-97e4-d95063a30429 None None] PATH_INFO: `/v3` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:58.353795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Tue Aug 30 14:36:58 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:05.951820 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cfa1b-b99a-4e52-8e69-e433ab5ee6f8 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:05.952323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cfa1b-b99a-4e52-8e69-e433ab5ee6f8 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:05.952423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cfa1b-b99a-4e52-8e69-e433ab5ee6f8 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:05.956659 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:37:05.961837 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc06379-2206-4cfb-9fe7-12ea4e3fd9bb None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:05.962227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc06379-2206-4cfb-9fe7-12ea4e3fd9bb None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:05.962378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc06379-2206-4cfb-9fe7-12ea4e3fd9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:06.031326 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: INFO dbcounter [None req-4cc06379-2206-4cfb-9fe7-12ea4e3fd9bb None None] Registered counter for database keystone Aug 30 14:37:06.052347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG oslo_db.sqlalchemy.engines [None req-4cc06379-2206-4cfb-9fe7-12ea4e3fd9bb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88059) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 14:37:06.058465 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writer thread running {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 14:37:06.235448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG passlib.handlers.bcrypt [None req-4cc06379-2206-4cfb-9fe7-12ea4e3fd9bb None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88059) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 14:37:06.235839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG passlib.handlers.bcrypt [None req-4cc06379-2206-4cfb-9fe7-12ea4e3fd9bb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88059) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 14:37:06.295328 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4cc06379-2206-4cfb-9fe7-12ea4e3fd9bb None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:06.296944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4cc06379-2206-4cfb-9fe7-12ea4e3fd9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:06.425656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 14:37:06.948246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cfa1b-b99a-4e52-8e69-e433ab5ee6f8 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:07.020122 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: INFO dbcounter [None req-9f9cfa1b-b99a-4e52-8e69-e433ab5ee6f8 None None] Registered counter for database keystone Aug 30 14:37:07.038895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG oslo_db.sqlalchemy.engines [None req-9f9cfa1b-b99a-4e52-8e69-e433ab5ee6f8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88058) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 14:37:07.045203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writer thread running {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 14:37:07.164528 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cfa1b-b99a-4e52-8e69-e433ab5ee6f8 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:07.168862 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301e9df-37fd-4721-b2c9-fe022204a02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:07.169481 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301e9df-37fd-4721-b2c9-fe022204a02c admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:07.169764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301e9df-37fd-4721-b2c9-fe022204a02c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:07.170035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301e9df-37fd-4721-b2c9-fe022204a02c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:07.178727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301e9df-37fd-4721-b2c9-fe022204a02c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:07.229113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4301e9df-37fd-4721-b2c9-fe022204a02c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 14:37:07.229460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301e9df-37fd-4721-b2c9-fe022204a02c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:07.239690 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1324 bytes} [Tue Aug 30 14:37:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 14:37:07.246994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc06379-2206-4cfb-9fe7-12ea4e3fd9bb None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:07.261143 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc06379-2206-4cfb-9fe7-12ea4e3fd9bb None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:07.266226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884534b1-a1d6-4e4b-9205-cafcad80daf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:07.267350 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884534b1-a1d6-4e4b-9205-cafcad80daf9 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:07.267629 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884534b1-a1d6-4e4b-9205-cafcad80daf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:07.267894 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884534b1-a1d6-4e4b-9205-cafcad80daf9 admin admin] PATH_INFO: `/v3/projects/d707a58a86f74ccbb5ffb499d9409139` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:07.269451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884534b1-a1d6-4e4b-9205-cafcad80daf9 admin admin] RBAC: Authorizing `identity:get_project(project_id=d707a58a86f74ccbb5ffb499d9409139)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:07.321937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: WARNING keystone.common.rbac_enforcer.enforcer [None req-884534b1-a1d6-4e4b-9205-cafcad80daf9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 14:37:07.322282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884534b1-a1d6-4e4b-9205-cafcad80daf9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:07.326872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 3/5] 10.222.0.27 () {56 vars in 1207 bytes} [Tue Aug 30 14:37:07 2022] GET /identity/v3/projects/d707a58a86f74ccbb5ffb499d9409139 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:09.204892 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee151da-7a3b-437c-b6bb-ff36e2e6359d None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:09.205419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee151da-7a3b-437c-b6bb-ff36e2e6359d None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:09.205419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee151da-7a3b-437c-b6bb-ff36e2e6359d None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:09.209516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 3/6] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:09.214790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d736ee-57fa-4c7f-9449-07a7283767b5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:09.215171 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d736ee-57fa-4c7f-9449-07a7283767b5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:09.215395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d736ee-57fa-4c7f-9449-07a7283767b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:09.236469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-19d736ee-57fa-4c7f-9449-07a7283767b5 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:09.238049 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-19d736ee-57fa-4c7f-9449-07a7283767b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:09.245942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 14:37:09.509434 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee151da-7a3b-437c-b6bb-ff36e2e6359d None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:09.521872 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee151da-7a3b-437c-b6bb-ff36e2e6359d None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:09.525975 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e513c-eb52-4419-a3dd-ff8f954a4227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:09.526564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e513c-eb52-4419-a3dd-ff8f954a4227 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:09.526839 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e513c-eb52-4419-a3dd-ff8f954a4227 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:09.527124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e513c-eb52-4419-a3dd-ff8f954a4227 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:09.535496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e513c-eb52-4419-a3dd-ff8f954a4227 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:09.537253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e513c-eb52-4419-a3dd-ff8f954a4227 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:09.545713 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1324 bytes} [Tue Aug 30 14:37:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:37:09.550847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d736ee-57fa-4c7f-9449-07a7283767b5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:09.559926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d736ee-57fa-4c7f-9449-07a7283767b5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:09.564183 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124ce90-2db9-434d-9b37-24d400098c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:09.564799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124ce90-2db9-434d-9b37-24d400098c14 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:09.565075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124ce90-2db9-434d-9b37-24d400098c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:09.565342 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124ce90-2db9-434d-9b37-24d400098c14 admin admin] PATH_INFO: `/v3/users/ef040204e84943a7ad9a173f7dd9bb2b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:09.566624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e124ce90-2db9-434d-9b37-24d400098c14 admin admin] RBAC: Authorizing `identity:get_user(user_id=ef040204e84943a7ad9a173f7dd9bb2b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:09.568485 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e124ce90-2db9-434d-9b37-24d400098c14 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:09.569977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 5/9] 10.222.0.27 () {56 vars in 1198 bytes} [Tue Aug 30 14:37:09 2022] GET /identity/v3/users/ef040204e84943a7ad9a173f7dd9bb2b => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:11.523168 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc982eb-d4cf-45a6-a413-ffd856632abc None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:11.523686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc982eb-d4cf-45a6-a413-ffd856632abc None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:11.523686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc982eb-d4cf-45a6-a413-ffd856632abc None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:11.524991 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:11.530311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164c592-3954-4595-ad62-9b4de394b20f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:11.530718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164c592-3954-4595-ad62-9b4de394b20f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:11.530859 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164c592-3954-4595-ad62-9b4de394b20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:11.548474 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6164c592-3954-4595-ad62-9b4de394b20f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:11.549632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6164c592-3954-4595-ad62-9b4de394b20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:11.564663 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea528a3-2c15-48e1-bc19-cb9c188ce350 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:11.564950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea528a3-2c15-48e1-bc19-cb9c188ce350 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:11.565224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea528a3-2c15-48e1-bc19-cb9c188ce350 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:11.566544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 6/11] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:11.571480 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931c536-c65b-4cc1-b3b6-9a68d1f118e7 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:11.571766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931c536-c65b-4cc1-b3b6-9a68d1f118e7 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:11.572400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931c536-c65b-4cc1-b3b6-9a68d1f118e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:11.613104 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG passlib.handlers.bcrypt [None req-e931c536-c65b-4cc1-b3b6-9a68d1f118e7 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88058) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 14:37:11.613484 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG passlib.handlers.bcrypt [None req-e931c536-c65b-4cc1-b3b6-9a68d1f118e7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88058) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 14:37:11.630440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 6/12] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:11.662966 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e931c536-c65b-4cc1-b3b6-9a68d1f118e7 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:11.664610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e931c536-c65b-4cc1-b3b6-9a68d1f118e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:11.762233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 7/13] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:37:11.893178 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164c592-3954-4595-ad62-9b4de394b20f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:11.918126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:11.994533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164c592-3954-4595-ad62-9b4de394b20f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:12.025213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931c536-c65b-4cc1-b3b6-9a68d1f118e7 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:12.029474 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb168e7-88d9-4df3-9f42-9412f9597cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:12.030104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb168e7-88d9-4df3-9f42-9412f9597cb5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:12.030382 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb168e7-88d9-4df3-9f42-9412f9597cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:12.030651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb168e7-88d9-4df3-9f42-9412f9597cb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:12.042375 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb168e7-88d9-4df3-9f42-9412f9597cb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:12.051262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-4eb168e7-88d9-4df3-9f42-9412f9597cb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:12.051262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:12.051262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:12.051262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:12.051262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:12.051262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:12.051262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:12.051262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:12.051262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:12.051262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:12.051262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:12.051262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:12.051262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:12.051262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:12.051262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:12.051262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:12.052286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:12.052286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:12.052286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:12.052286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:12.052286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:12.052286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:12.052286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:12.052286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:12.052286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:12.052286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:12.052286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:12.052286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:12.052286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:12.052286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:12.052286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:12.052286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:12.053206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:12.053206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:12.053206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:12.053206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:12.053206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:12.053206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:12.053206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:12.053206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:12.053206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:12.053206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:12.053206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:12.053206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:12.053206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 7/14] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:37:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:12.057918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb168e7-88d9-4df3-9f42-9412f9597cb5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:12.130261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931c536-c65b-4cc1-b3b6-9a68d1f118e7 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:12.132551 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:12.136812 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:12.154812 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb168e7-88d9-4df3-9f42-9412f9597cb5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:12.167575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae272366-e246-421f-bab8-e65d0b7df263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:12.168426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae272366-e246-421f-bab8-e65d0b7df263 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:12.168706 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae272366-e246-421f-bab8-e65d0b7df263 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:12.168974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae272366-e246-421f-bab8-e65d0b7df263 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:12.188848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6db10-2177-413e-9051-00ef2842dd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:12.189467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6db10-2177-413e-9051-00ef2842dd53 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:12.189746 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6db10-2177-413e-9051-00ef2842dd53 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:12.190016 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6db10-2177-413e-9051-00ef2842dd53 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:12.190691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6db10-2177-413e-9051-00ef2842dd53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:12.190989 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6db10-2177-413e-9051-00ef2842dd53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:12.192423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6db10-2177-413e-9051-00ef2842dd53 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:12.204471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 8/15] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:12.209929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6db10-2177-413e-9051-00ef2842dd53 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:12.237053 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae272366-e246-421f-bab8-e65d0b7df263 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:12.238588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae272366-e246-421f-bab8-e65d0b7df263 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:12.312134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6db10-2177-413e-9051-00ef2842dd53 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:12.338822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 8/16] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 14:37:12.344310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae272366-e246-421f-bab8-e65d0b7df263 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:12.345812 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28989676-5c40-4323-93dd-c11cc58a215b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:12.346605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28989676-5c40-4323-93dd-c11cc58a215b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:12.346978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:12.347827 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28989676-5c40-4323-93dd-c11cc58a215b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:12.347940 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28989676-5c40-4323-93dd-c11cc58a215b None admin] PATH_INFO: `/v3/users/ef040204e84943a7ad9a173f7dd9bb2b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:12.349075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28989676-5c40-4323-93dd-c11cc58a215b None admin] RBAC: Authorizing `identity:get_user(user_id=ef040204e84943a7ad9a173f7dd9bb2b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:12.350533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28989676-5c40-4323-93dd-c11cc58a215b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:12.352131 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 9/17] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:12 2022] GET /identity/v3/users/ef040204e84943a7ad9a173f7dd9bb2b => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:12.357554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28989676-5c40-4323-93dd-c11cc58a215b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:12.373254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:12.442956 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae272366-e246-421f-bab8-e65d0b7df263 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:12.458494 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28989676-5c40-4323-93dd-c11cc58a215b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:12.477275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f85071-d58a-42fb-a33e-2a6022885292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:12.478200 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f85071-d58a-42fb-a33e-2a6022885292 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:12.478486 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f85071-d58a-42fb-a33e-2a6022885292 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:12.478761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f85071-d58a-42fb-a33e-2a6022885292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:12.490567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6c8f0-7f5f-4bb4-b779-717b520fa108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:12.491218 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6c8f0-7f5f-4bb4-b779-717b520fa108 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:12.491496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6c8f0-7f5f-4bb4-b779-717b520fa108 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:12.491764 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6c8f0-7f5f-4bb4-b779-717b520fa108 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:12.492870 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6c8f0-7f5f-4bb4-b779-717b520fa108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:12.494310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6c8f0-7f5f-4bb4-b779-717b520fa108 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:12.496080 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 10/18] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:12.496830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f85071-d58a-42fb-a33e-2a6022885292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:12.498352 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f85071-d58a-42fb-a33e-2a6022885292 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:12.502071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6c8f0-7f5f-4bb4-b779-717b520fa108 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:12.517473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-18f85071-d58a-42fb-a33e-2a6022885292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:37:12.517473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:12.517473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:12.517473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:12.517473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:12.517473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:12.517473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:12.517473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:12.517473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:12.517473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:12.517473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:12.517473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:12.517473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:37:12.517473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:37:12.517473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:37:12.517473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:12.518401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:37:12.519225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:37:12.519225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:37:12.519225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:37:12.519225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 9/19] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:37:12.524127 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f85071-d58a-42fb-a33e-2a6022885292 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:12.580725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:12.581573 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:12.599959 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6c8f0-7f5f-4bb4-b779-717b520fa108 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:12.623202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f85071-d58a-42fb-a33e-2a6022885292 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:12.634369 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9f30a-e116-428a-b548-5ff3ab92b421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:12.634985 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9f30a-e116-428a-b548-5ff3ab92b421 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:12.635295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9f30a-e116-428a-b548-5ff3ab92b421 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:12.635571 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9f30a-e116-428a-b548-5ff3ab92b421 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:12.636302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db9f30a-e116-428a-b548-5ff3ab92b421 None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.domain.id=default, user.id=ef040204e84943a7ad9a173f7dd9bb2b) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:12.636608 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db9f30a-e116-428a-b548-5ff3ab92b421 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:12.638051 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db9f30a-e116-428a-b548-5ff3ab92b421 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:12.644513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 11/20] 10.222.0.27 () {56 vars in 1322 bytes} [Tue Aug 30 14:37:12 2022] GET /identity/v3/role_assignments?user.id=ef040204e84943a7ad9a173f7dd9bb2b&scope.domain.id=default&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:12.657954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8ab03-b044-4c0e-94a8-6b8f9c930b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:12.658598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8ab03-b044-4c0e-94a8-6b8f9c930b1a None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:12.658876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8ab03-b044-4c0e-94a8-6b8f9c930b1a None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:12.659176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8ab03-b044-4c0e-94a8-6b8f9c930b1a None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:12.659851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8ab03-b044-4c0e-94a8-6b8f9c930b1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:12.660152 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8ab03-b044-4c0e-94a8-6b8f9c930b1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:12.661599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8ab03-b044-4c0e-94a8-6b8f9c930b1a None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:12.681650 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 10/21] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:14.663965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd90713-b999-412a-baf3-0dffa0bb3fdd None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:14.664484 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd90713-b999-412a-baf3-0dffa0bb3fdd None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:14.664484 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd90713-b999-412a-baf3-0dffa0bb3fdd None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:14.665788 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 12/22] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:14.671033 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb72f89-facc-4dc3-b166-53c4700a49f4 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:14.671458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb72f89-facc-4dc3-b166-53c4700a49f4 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:14.672219 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb72f89-facc-4dc3-b166-53c4700a49f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:14.700488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-dfb72f89-facc-4dc3-b166-53c4700a49f4 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:14.701702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-dfb72f89-facc-4dc3-b166-53c4700a49f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:14.797448 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 11/23] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:15.178325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730646e-a109-4491-938e-996db710b462 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:15.178706 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730646e-a109-4491-938e-996db710b462 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:15.178892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730646e-a109-4491-938e-996db710b462 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:15.180281 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 13/24] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:15.192762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f7bc3-6fdd-44ed-8123-5e53c0e6ff45 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:15.193056 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f7bc3-6fdd-44ed-8123-5e53c0e6ff45 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:15.193331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f7bc3-6fdd-44ed-8123-5e53c0e6ff45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:15.220140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-f25f7bc3-6fdd-44ed-8123-5e53c0e6ff45 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:15.221642 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-f25f7bc3-6fdd-44ed-8123-5e53c0e6ff45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:15.222827 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d92d6f-9750-4321-b070-17e6b35fd019 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:15.223126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d92d6f-9750-4321-b070-17e6b35fd019 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:15.223435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d92d6f-9750-4321-b070-17e6b35fd019 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:15.224781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 14/25] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:15.230913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734eb802-91a8-4af8-80e8-2139f8aca71e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:15.231243 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734eb802-91a8-4af8-80e8-2139f8aca71e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:15.231522 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734eb802-91a8-4af8-80e8-2139f8aca71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:15.257698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-734eb802-91a8-4af8-80e8-2139f8aca71e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:15.259242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-734eb802-91a8-4af8-80e8-2139f8aca71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:15.305761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:15.344942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 12/26] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:15.346685 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7fc52-4f0f-419b-958b-e4a827c1d188 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:15.347293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7fc52-4f0f-419b-958b-e4a827c1d188 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:15.347766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7fc52-4f0f-419b-958b-e4a827c1d188 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:15.349195 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 13/27] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:15.354196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7fc52-4f0f-419b-958b-e4a827c1d188 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:15.362371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 15/28] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:37:15.371122 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492fc18-2cda-41f8-b068-5d5d14887a04 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:15.371122 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492fc18-2cda-41f8-b068-5d5d14887a04 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:15.371122 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492fc18-2cda-41f8-b068-5d5d14887a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:15.393524 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-d492fc18-2cda-41f8-b068-5d5d14887a04 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:15.394730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d492fc18-2cda-41f8-b068-5d5d14887a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:15.493465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7fc52-4f0f-419b-958b-e4a827c1d188 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:15.531980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002cd527-b89a-4d13-8d33-4cb2cbab7278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:15.532693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cd527-b89a-4d13-8d33-4cb2cbab7278 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:15.532980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cd527-b89a-4d13-8d33-4cb2cbab7278 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:15.533250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cd527-b89a-4d13-8d33-4cb2cbab7278 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:15.546653 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002cd527-b89a-4d13-8d33-4cb2cbab7278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:15.553240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-002cd527-b89a-4d13-8d33-4cb2cbab7278 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:15.553240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:15.553240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:15.553240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:15.553240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:15.553240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:15.553240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:15.553240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:15.553240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:15.553240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:15.553240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:15.553240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:15.553240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:15.553240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:15.553240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:15.553240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:15.554081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:15.554081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:15.554081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:15.554081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:15.554081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:15.554081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:15.554081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:15.554081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:15.554081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:15.554081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:15.554081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:15.554081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:15.554081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:15.554081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:15.554081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:15.554081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:15.554926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:15.554926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:15.554926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:15.554926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:15.554926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:15.554926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:15.554926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:15.554926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:15.554926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:15.554926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:15.554926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:15.554926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:37:15.554926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 14/29] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:37:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:15.560383 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002cd527-b89a-4d13-8d33-4cb2cbab7278 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:15.561489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 16/30] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:15.600563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:15.672777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002cd527-b89a-4d13-8d33-4cb2cbab7278 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:15.708598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d12db-520e-44a3-bc1d-7a748302f16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:15.709203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d12db-520e-44a3-bc1d-7a748302f16d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:15.709535 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d12db-520e-44a3-bc1d-7a748302f16d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:15.709859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d12db-520e-44a3-bc1d-7a748302f16d None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:15.710536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d12db-520e-44a3-bc1d-7a748302f16d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:15.710840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d12db-520e-44a3-bc1d-7a748302f16d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:15.712323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7d12db-520e-44a3-bc1d-7a748302f16d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:15.725393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 15/31] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:15.730999 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492fc18-2cda-41f8-b068-5d5d14887a04 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:15.743399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:15.840853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d12db-520e-44a3-bc1d-7a748302f16d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:15.874780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492fc18-2cda-41f8-b068-5d5d14887a04 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:15.972986 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:15.975804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc01553-77d6-4f9b-bc01-4dc6ee845234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:15.976469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc01553-77d6-4f9b-bc01-4dc6ee845234 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:15.976762 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc01553-77d6-4f9b-bc01-4dc6ee845234 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:15.977044 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc01553-77d6-4f9b-bc01-4dc6ee845234 None admin] PATH_INFO: `/v3/users/ef040204e84943a7ad9a173f7dd9bb2b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:15.977338 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d12db-520e-44a3-bc1d-7a748302f16d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:15.978230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc01553-77d6-4f9b-bc01-4dc6ee845234 None admin] RBAC: Authorizing `identity:get_user(user_id=ef040204e84943a7ad9a173f7dd9bb2b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:15.979759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc01553-77d6-4f9b-bc01-4dc6ee845234 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:15.981610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 17/32] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:15 2022] GET /identity/v3/users/ef040204e84943a7ad9a173f7dd9bb2b => generated 247 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:15.982198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc01553-77d6-4f9b-bc01-4dc6ee845234 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:16.021421 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d69a5-b5e9-4112-b1ba-3c86b0b42048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:16.022124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d69a5-b5e9-4112-b1ba-3c86b0b42048 None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:16.022408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d69a5-b5e9-4112-b1ba-3c86b0b42048 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:16.022680 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d69a5-b5e9-4112-b1ba-3c86b0b42048 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:16.023394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d69a5-b5e9-4112-b1ba-3c86b0b42048 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:16.024784 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d69a5-b5e9-4112-b1ba-3c86b0b42048 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:16.067965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:16.098910 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc01553-77d6-4f9b-bc01-4dc6ee845234 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:16.134464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f183e8b-f1a0-4f9a-b198-226f9b99b79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:16.135352 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f183e8b-f1a0-4f9a-b198-226f9b99b79b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:16.135766 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f183e8b-f1a0-4f9a-b198-226f9b99b79b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:16.136227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f183e8b-f1a0-4f9a-b198-226f9b99b79b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:16.148499 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f183e8b-f1a0-4f9a-b198-226f9b99b79b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:16.150285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f183e8b-f1a0-4f9a-b198-226f9b99b79b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:16.167052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-0f183e8b-f1a0-4f9a-b198-226f9b99b79b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:37:16.167052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:16.167052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:16.167052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:16.167052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:16.167052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:16.167052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:16.167052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:16.167052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:16.167052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:16.167052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:16.167052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:16.167052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:37:16.167052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:37:16.167052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:37:16.167052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:16.168145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:37:16.169127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:37:16.169127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:37:16.169127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:16.170597 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 18/33] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:16.171634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f183e8b-f1a0-4f9a-b198-226f9b99b79b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:16.242907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 16/34] 10.222.0.27 () {60 vars in 1131 bytes} [Tue Aug 30 14:37:15 2022] POST /identity/v3/roles => generated 219 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:37:16.244215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d69a5-b5e9-4112-b1ba-3c86b0b42048 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:16.304192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f183e8b-f1a0-4f9a-b198-226f9b99b79b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:16.350208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:16.375747 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839cea2-df4a-4c38-a9bb-20914ffb13b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:16.376377 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839cea2-df4a-4c38-a9bb-20914ffb13b1 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:16.376657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839cea2-df4a-4c38-a9bb-20914ffb13b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:16.376931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839cea2-df4a-4c38-a9bb-20914ffb13b1 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:16.377634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839cea2-df4a-4c38-a9bb-20914ffb13b1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:16.379060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839cea2-df4a-4c38-a9bb-20914ffb13b1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:16.413090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d69a5-b5e9-4112-b1ba-3c86b0b42048 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:16.460733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a89fe-80cf-47a6-a051-77a74a89e3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:16.461380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a89fe-80cf-47a6-a051-77a74a89e3b8 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:16.461661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a89fe-80cf-47a6-a051-77a74a89e3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:16.461935 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a89fe-80cf-47a6-a051-77a74a89e3b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:16.463553 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a89fe-80cf-47a6-a051-77a74a89e3b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:16.465241 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a89fe-80cf-47a6-a051-77a74a89e3b8 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:16.466839 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 17/35] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:16.467647 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a89fe-80cf-47a6-a051-77a74a89e3b8 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:16.508893 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:16.579503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 19/36] 10.222.0.27 () {60 vars in 1131 bytes} [Tue Aug 30 14:37:16 2022] POST /identity/v3/roles => generated 225 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:37:16.580391 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839cea2-df4a-4c38-a9bb-20914ffb13b1 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:16.605737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a89fe-80cf-47a6-a051-77a74a89e3b8 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:16.654558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e62b8-24e4-4fe1-a449-8fb0eff1e366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:16.655210 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e62b8-24e4-4fe1-a449-8fb0eff1e366 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:16.655496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e62b8-24e4-4fe1-a449-8fb0eff1e366 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:16.655828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e62b8-24e4-4fe1-a449-8fb0eff1e366 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:16.657006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e62b8-24e4-4fe1-a449-8fb0eff1e366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:16.657322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e62b8-24e4-4fe1-a449-8fb0eff1e366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:16.658755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e62b8-24e4-4fe1-a449-8fb0eff1e366 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:16.671344 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 18/37] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:16.674073 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734843a-b422-49fa-ab30-7261454e17dc None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:16.674667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734843a-b422-49fa-ab30-7261454e17dc None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:16.674667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734843a-b422-49fa-ab30-7261454e17dc None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:16.676005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 19/38] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:16.677042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ccaa7-6168-46f4-b401-c69ac0365fc3 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:16.677320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ccaa7-6168-46f4-b401-c69ac0365fc3 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:16.677591 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ccaa7-6168-46f4-b401-c69ac0365fc3 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:16.678810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 20/39] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:16.680052 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ccaa7-6168-46f4-b401-c69ac0365fc3 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:16.708568 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839cea2-df4a-4c38-a9bb-20914ffb13b1 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:16.749283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7de4f-0ac9-49c4-a439-1ff9dbf7c4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:16.750476 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7de4f-0ac9-49c4-a439-1ff9dbf7c4a1 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:16.750756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7de4f-0ac9-49c4-a439-1ff9dbf7c4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:16.751031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7de4f-0ac9-49c4-a439-1ff9dbf7c4a1 None admin] PATH_INFO: `/v3/domains/default/users/ef040204e84943a7ad9a173f7dd9bb2b/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:16.753363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7de4f-0ac9-49c4-a439-1ff9dbf7c4a1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ef040204e84943a7ad9a173f7dd9bb2b, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:16.754938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7de4f-0ac9-49c4-a439-1ff9dbf7c4a1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:16.781568 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:16.797247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ccaa7-6168-46f4-b401-c69ac0365fc3 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:16.830946 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f48992-9d91-4516-af00-f626e12ce583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:16.831577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f48992-9d91-4516-af00-f626e12ce583 None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:16.831857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f48992-9d91-4516-af00-f626e12ce583 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:16.832125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f48992-9d91-4516-af00-f626e12ce583 None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:16.832768 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f48992-9d91-4516-af00-f626e12ce583 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:16.834212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f48992-9d91-4516-af00-f626e12ce583 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:16.892674 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 20/40] 10.222.0.27 () {58 vars in 1361 bytes} [Tue Aug 30 14:37:16 2022] PUT /identity/v3/domains/default/users/ef040204e84943a7ad9a173f7dd9bb2b/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:37:16.893924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba084b2-0637-4607-9ff7-101a22688228 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:16.894208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba084b2-0637-4607-9ff7-101a22688228 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:16.894479 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba084b2-0637-4607-9ff7-101a22688228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:16.914317 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3ba084b2-0637-4607-9ff7-101a22688228 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:16.915951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3ba084b2-0637-4607-9ff7-101a22688228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:16.962817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:16.975159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 21/41] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 30 14:37:16 2022] POST /identity/v3/projects => generated 301 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:37:16.976804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf8315-1c37-4655-b4d0-ab61c81c0e2d None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:16.977158 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf8315-1c37-4655-b4d0-ab61c81c0e2d None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:16.977539 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf8315-1c37-4655-b4d0-ab61c81c0e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:17.005142 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 21/42] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:17.027637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-adaf8315-1c37-4655-b4d0-ab61c81c0e2d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:17.029917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-adaf8315-1c37-4655-b4d0-ab61c81c0e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:17.109930 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 22/43] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:17.277969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba084b2-0637-4607-9ff7-101a22688228 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:17.375391 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf8315-1c37-4655-b4d0-ab61c81c0e2d None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:17.380932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba084b2-0637-4607-9ff7-101a22688228 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:17.450993 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:17.488656 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf8315-1c37-4655-b4d0-ab61c81c0e2d None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:17.525605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df52169-6187-484a-a6bf-3603295d9bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:17.526269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9944d-74fa-4d77-a38d-0458faf9e947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:17.526792 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df52169-6187-484a-a6bf-3603295d9bb6 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:17.526915 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9944d-74fa-4d77-a38d-0458faf9e947 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:17.527308 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9944d-74fa-4d77-a38d-0458faf9e947 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:17.527580 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9944d-74fa-4d77-a38d-0458faf9e947 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:17.528668 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9944d-74fa-4d77-a38d-0458faf9e947 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:17.529475 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df52169-6187-484a-a6bf-3603295d9bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:17.529937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df52169-6187-484a-a6bf-3603295d9bb6 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:17.530377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9944d-74fa-4d77-a38d-0458faf9e947 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:17.531290 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df52169-6187-484a-a6bf-3603295d9bb6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:17.532025 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 23/44] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:17.533103 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df52169-6187-484a-a6bf-3603295d9bb6 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:17.537594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9944d-74fa-4d77-a38d-0458faf9e947 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:17.614124 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 22/45] 10.222.0.27 () {60 vars in 1131 bytes} [Tue Aug 30 14:37:17 2022] POST /identity/v3/roles => generated 223 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:37:17.753591 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9944d-74fa-4d77-a38d-0458faf9e947 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:17.790640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d34ff-a78a-43ce-8532-46da4750d9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:17.791221 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d34ff-a78a-43ce-8532-46da4750d9fd None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:17.791505 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d34ff-a78a-43ce-8532-46da4750d9fd None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:17.791780 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d34ff-a78a-43ce-8532-46da4750d9fd None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:17.792516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d34ff-a78a-43ce-8532-46da4750d9fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:17.793955 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d34ff-a78a-43ce-8532-46da4750d9fd None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:17.799485 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:17.893935 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 24/46] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 30 14:37:17 2022] POST /identity/v3/projects => generated 312 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:37:18.825446 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff2965-0878-47bb-8b8a-72cddecbc3b9 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:18.825953 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff2965-0878-47bb-8b8a-72cddecbc3b9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:18.826033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff2965-0878-47bb-8b8a-72cddecbc3b9 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:18.827360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 23/47] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:18.833035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7735b3-7a75-4adb-9f4c-e5658768d393 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:18.833402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7735b3-7a75-4adb-9f4c-e5658768d393 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:18.833638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7735b3-7a75-4adb-9f4c-e5658768d393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:18.852434 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-7c7735b3-7a75-4adb-9f4c-e5658768d393 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:18.853615 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7c7735b3-7a75-4adb-9f4c-e5658768d393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:18.918603 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 25/48] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:19.182168 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff2965-0878-47bb-8b8a-72cddecbc3b9 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:19.217896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:19.285740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff2965-0878-47bb-8b8a-72cddecbc3b9 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:19.318991 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29482ab4-c8de-47ad-97ac-f21f5ba3c215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:19.319647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29482ab4-c8de-47ad-97ac-f21f5ba3c215 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:19.319929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29482ab4-c8de-47ad-97ac-f21f5ba3c215 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:19.320201 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29482ab4-c8de-47ad-97ac-f21f5ba3c215 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:19.330361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29482ab4-c8de-47ad-97ac-f21f5ba3c215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:19.335137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-29482ab4-c8de-47ad-97ac-f21f5ba3c215 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:19.335137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:19.335137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:19.335137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:19.335137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:19.335137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:19.335137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:19.335137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:19.335137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:19.335137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:19.335137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:19.335137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:19.335137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:19.335137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:19.335137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:19.335137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:19.336092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:19.336092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:19.336092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:19.336092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:19.336092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:19.336092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:19.336092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:19.336092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:19.336092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:19.336092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:19.336092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:19.336092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:19.336092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:19.336092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:19.336092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:19.336092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:19.337155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:19.337155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:19.337155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:19.337155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:19.337155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:19.337155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:19.337155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:19.337155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:19.337155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:19.337155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:19.337155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:19.337155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:19.337155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 24/49] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:37:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:19.342050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7735b3-7a75-4adb-9f4c-e5658768d393 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:19.438124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7735b3-7a75-4adb-9f4c-e5658768d393 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:19.454455 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:19.470781 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f8bd4-52a9-4993-97bd-d16cc58f2d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:19.471427 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f8bd4-52a9-4993-97bd-d16cc58f2d73 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:19.471706 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f8bd4-52a9-4993-97bd-d16cc58f2d73 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:19.471974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f8bd4-52a9-4993-97bd-d16cc58f2d73 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:19.472645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f8bd4-52a9-4993-97bd-d16cc58f2d73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:19.472946 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f8bd4-52a9-4993-97bd-d16cc58f2d73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:19.474429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f8bd4-52a9-4993-97bd-d16cc58f2d73 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:19.482603 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 26/50] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:19.488705 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29482ab4-c8de-47ad-97ac-f21f5ba3c215 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:19.580049 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:19.586482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29482ab4-c8de-47ad-97ac-f21f5ba3c215 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:19.619802 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19909791-9c0d-4414-966a-3065a90f9431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:19.620426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19909791-9c0d-4414-966a-3065a90f9431 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:19.620709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19909791-9c0d-4414-966a-3065a90f9431 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:19.620983 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19909791-9c0d-4414-966a-3065a90f9431 None admin] PATH_INFO: `/v3/users/ef040204e84943a7ad9a173f7dd9bb2b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:19.622111 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19909791-9c0d-4414-966a-3065a90f9431 None admin] RBAC: Authorizing `identity:get_user(user_id=ef040204e84943a7ad9a173f7dd9bb2b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:19.623618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19909791-9c0d-4414-966a-3065a90f9431 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:19.625230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 25/51] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:19 2022] GET /identity/v3/users/ef040204e84943a7ad9a173f7dd9bb2b => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:19.630930 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f8bd4-52a9-4993-97bd-d16cc58f2d73 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:19.727864 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f8bd4-52a9-4993-97bd-d16cc58f2d73 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:19.767993 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb317bd-98b4-4f3a-bcf1-1b434c2dc2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:19.768625 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb317bd-98b4-4f3a-bcf1-1b434c2dc2f3 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:19.768906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb317bd-98b4-4f3a-bcf1-1b434c2dc2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:19.769181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb317bd-98b4-4f3a-bcf1-1b434c2dc2f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:19.770312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb317bd-98b4-4f3a-bcf1-1b434c2dc2f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:19.771835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb317bd-98b4-4f3a-bcf1-1b434c2dc2f3 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:19.773418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 27/52] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:19.780120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19909791-9c0d-4414-966a-3065a90f9431 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:19.788692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f6d26-f657-4d36-9674-497cd76badf8 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:19.788979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f6d26-f657-4d36-9674-497cd76badf8 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:19.789251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f6d26-f657-4d36-9674-497cd76badf8 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:19.790555 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 28/53] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:37:19.795505 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f95961-b4d5-4300-8ad9-f0cad2969155 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:19.795792 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f95961-b4d5-4300-8ad9-f0cad2969155 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:19.796069 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f95961-b4d5-4300-8ad9-f0cad2969155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:19.813966 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-32f95961-b4d5-4300-8ad9-f0cad2969155 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:19.815200 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-32f95961-b4d5-4300-8ad9-f0cad2969155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:19.856924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:19.879703 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19909791-9c0d-4414-966a-3065a90f9431 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:19.880154 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 29/54] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:37:19.913545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac27044-0aae-44d2-b4b6-11653c334dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:19.914147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac27044-0aae-44d2-b4b6-11653c334dd2 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:19.914434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac27044-0aae-44d2-b4b6-11653c334dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:19.914713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac27044-0aae-44d2-b4b6-11653c334dd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:19.915480 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac27044-0aae-44d2-b4b6-11653c334dd2 None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.domain.id=default, user.id=ef040204e84943a7ad9a173f7dd9bb2b) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:19.915794 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac27044-0aae-44d2-b4b6-11653c334dd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:19.917263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac27044-0aae-44d2-b4b6-11653c334dd2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:19.923402 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 26/55] 10.222.0.27 () {56 vars in 1322 bytes} [Tue Aug 30 14:37:19 2022] GET /identity/v3/role_assignments?user.id=ef040204e84943a7ad9a173f7dd9bb2b&scope.domain.id=default&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:20.151757 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f95961-b4d5-4300-8ad9-f0cad2969155 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:20.246872 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f95961-b4d5-4300-8ad9-f0cad2969155 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:20.281448 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ff42c-8418-4f8a-8da3-4e22793789bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:20.282084 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ff42c-8418-4f8a-8da3-4e22793789bd None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:20.282368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ff42c-8418-4f8a-8da3-4e22793789bd None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:20.282652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ff42c-8418-4f8a-8da3-4e22793789bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:20.283779 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ff42c-8418-4f8a-8da3-4e22793789bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:20.285240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ff42c-8418-4f8a-8da3-4e22793789bd None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:20.286832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 30/56] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:20.292528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac27044-0aae-44d2-b4b6-11653c334dd2 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:20.319659 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:20.390511 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac27044-0aae-44d2-b4b6-11653c334dd2 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:20.424671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6adfa3-2baa-418a-a20a-89216a9a6f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:20.425204 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6adfa3-2baa-418a-a20a-89216a9a6f71 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:20.425486 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6adfa3-2baa-418a-a20a-89216a9a6f71 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:20.425760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6adfa3-2baa-418a-a20a-89216a9a6f71 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:20.426420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6adfa3-2baa-418a-a20a-89216a9a6f71 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:20.427886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6adfa3-2baa-418a-a20a-89216a9a6f71 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:20.509943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 27/57] 10.222.0.27 () {60 vars in 1140 bytes} [Tue Aug 30 14:37:20 2022] POST /identity/v3/projects => generated 298 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:37:22.400328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5633f-1af4-42d8-9a25-07176ac1352e None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:22.400858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5633f-1af4-42d8-9a25-07176ac1352e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:22.400858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5633f-1af4-42d8-9a25-07176ac1352e None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:22.402226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 31/58] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:22.407615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4acec-ff91-40ee-8075-066ac13b049a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:22.408009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4acec-ff91-40ee-8075-066ac13b049a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:22.408176 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4acec-ff91-40ee-8075-066ac13b049a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:22.426514 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-d2d4acec-ff91-40ee-8075-066ac13b049a None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:22.427784 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d2d4acec-ff91-40ee-8075-066ac13b049a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:22.492293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 28/59] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:22.762072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5633f-1af4-42d8-9a25-07176ac1352e None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:22.809255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:22.860925 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5633f-1af4-42d8-9a25-07176ac1352e None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:22.892407 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668b86f-5e1e-487c-9bce-d687dc88dd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:22.893023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668b86f-5e1e-487c-9bce-d687dc88dd95 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:22.893303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668b86f-5e1e-487c-9bce-d687dc88dd95 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:22.893576 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668b86f-5e1e-487c-9bce-d687dc88dd95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:22.894694 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668b86f-5e1e-487c-9bce-d687dc88dd95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:22.896162 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668b86f-5e1e-487c-9bce-d687dc88dd95 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:22.897723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 32/60] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:22.903159 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4acec-ff91-40ee-8075-066ac13b049a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:22.913811 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:23.004925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4acec-ff91-40ee-8075-066ac13b049a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:23.038075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87f273-dc01-4a1f-936f-be0686e04bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:23.038614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87f273-dc01-4a1f-936f-be0686e04bb8 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:23.038891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87f273-dc01-4a1f-936f-be0686e04bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:23.039184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87f273-dc01-4a1f-936f-be0686e04bb8 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:23.039837 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87f273-dc01-4a1f-936f-be0686e04bb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:23.041270 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87f273-dc01-4a1f-936f-be0686e04bb8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:23.148859 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 29/61] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:37:22 2022] POST /identity/v3/users => generated 275 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:37:26.149332 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da955314-60a6-4957-8744-0976a5f174ea None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:26.150271 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da955314-60a6-4957-8744-0976a5f174ea None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:26.150724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da955314-60a6-4957-8744-0976a5f174ea None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:26.152291 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 33/62] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:26.160521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1829a-5607-48a6-a7eb-5dca827a6f63 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:26.161425 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1829a-5607-48a6-a7eb-5dca827a6f63 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:26.161907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1829a-5607-48a6-a7eb-5dca827a6f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:26.183449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-20e1829a-5607-48a6-a7eb-5dca827a6f63 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:26.184953 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-20e1829a-5607-48a6-a7eb-5dca827a6f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:26.273701 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:26.284716 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 30/63] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:26.286739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f6048-c219-44ec-9d1c-c6483443c6b4 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:26.287030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f6048-c219-44ec-9d1c-c6483443c6b4 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:26.287333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f6048-c219-44ec-9d1c-c6483443c6b4 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:26.289248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 34/64] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:26.294222 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c0c7c-8d54-4929-8801-5948f078418e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:26.294522 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c0c7c-8d54-4929-8801-5948f078418e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:26.294799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c0c7c-8d54-4929-8801-5948f078418e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:26.324531 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-870c0c7c-8d54-4929-8801-5948f078418e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:26.329338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-870c0c7c-8d54-4929-8801-5948f078418e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:26.424695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 31/65] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:26.478436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b79a9-4f41-4c7d-983b-ad7fd56b516a None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:26.478747 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b79a9-4f41-4c7d-983b-ad7fd56b516a None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:26.478996 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b79a9-4f41-4c7d-983b-ad7fd56b516a None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:26.480377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 35/66] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:26.486255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c5311-12f4-4444-91f8-61d30340f049 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:26.486555 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c5311-12f4-4444-91f8-61d30340f049 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:26.486833 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c5311-12f4-4444-91f8-61d30340f049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:26.512114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-812c5311-12f4-4444-91f8-61d30340f049 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:26.513486 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-812c5311-12f4-4444-91f8-61d30340f049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:26.610057 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 32/67] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:26.614124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b79a9-4f41-4c7d-983b-ad7fd56b516a None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:26.620186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a3dfc-5e3e-4517-897e-7c8e5b666586 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:26.620474 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a3dfc-5e3e-4517-897e-7c8e5b666586 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:26.620748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a3dfc-5e3e-4517-897e-7c8e5b666586 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:26.622359 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 33/68] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:26.632653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5044b08-a63c-4282-ab7c-67262484f1a8 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:26.632942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5044b08-a63c-4282-ab7c-67262484f1a8 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:26.633225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5044b08-a63c-4282-ab7c-67262484f1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:26.667032 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c5044b08-a63c-4282-ab7c-67262484f1a8 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:26.668275 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c5044b08-a63c-4282-ab7c-67262484f1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:26.767722 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:26.786365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 34/69] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:37:26.797223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b79a9-4f41-4c7d-983b-ad7fd56b516a None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:26.821458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:26.828652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5044b08-a63c-4282-ab7c-67262484f1a8 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:26.850336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa3802-c64a-4cb8-b4d0-a459aa2cbcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:26.850979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa3802-c64a-4cb8-b4d0-a459aa2cbcaa None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:26.851357 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa3802-c64a-4cb8-b4d0-a459aa2cbcaa None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:26.851575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa3802-c64a-4cb8-b4d0-a459aa2cbcaa None admin] PATH_INFO: `/v3/roles/member` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:26.871093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa3802-c64a-4cb8-b4d0-a459aa2cbcaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:26.882279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-c2aa3802-c64a-4cb8-b4d0-a459aa2cbcaa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:26.882279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:26.882279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:26.882279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:26.882279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:26.882279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:26.882279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:26.882279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:26.882279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:26.882279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:26.882279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:26.882279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:26.882279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:26.882279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:26.882279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:26.882279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:26.883121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:26.883121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:26.883121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:26.883121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:26.883121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:26.883121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:26.883121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:26.883121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:26.883121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:26.883121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:26.883121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:26.883121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:26.883121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:26.883121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:26.883121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:26.883121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:26.883876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:26.883876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:26.883876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:26.883876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:26.883876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:26.883876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:26.883876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:26.883876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:26.883876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:26.883876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:26.883876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:26.883876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:37:26.885653 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 36/70] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:26.889323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa3802-c64a-4cb8-b4d0-a459aa2cbcaa None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:26.973422 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5044b08-a63c-4282-ab7c-67262484f1a8 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:27.000399 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa3802-c64a-4cb8-b4d0-a459aa2cbcaa None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:27.009227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4c78c-abc6-4c35-8b5e-e03489289723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:27.009858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4c78c-abc6-4c35-8b5e-e03489289723 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:27.010146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4c78c-abc6-4c35-8b5e-e03489289723 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:27.010413 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4c78c-abc6-4c35-8b5e-e03489289723 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:27.020438 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4c78c-abc6-4c35-8b5e-e03489289723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:27.025203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-81f4c78c-abc6-4c35-8b5e-e03489289723 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:27.025203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:27.025203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:27.025203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:27.025203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:27.025203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:27.025203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:27.025203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:27.025203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:27.025203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:27.025203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:27.025203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:27.025203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:27.025203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:27.025203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:27.025203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:27.026077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:27.026077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:27.026077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:27.026077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:27.026077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:27.026077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:27.026077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:27.026077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:27.026077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:27.026077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:27.026077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:27.026077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:27.026077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:27.026077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:27.026077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:27.026077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:27.026839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:27.026839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:27.026839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:27.026839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:27.026839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:27.026839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:27.026839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:27.026839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:27.026839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:27.026839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:27.026839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:27.026839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:27.026839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 35/71] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:37:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:37:27.027886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4c78c-abc6-4c35-8b5e-e03489289723 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:27.041736 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2416b4-0a88-482e-b4b9-130f0fc5a6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:27.042370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2416b4-0a88-482e-b4b9-130f0fc5a6bc None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:27.042654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2416b4-0a88-482e-b4b9-130f0fc5a6bc None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:27.042930 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2416b4-0a88-482e-b4b9-130f0fc5a6bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:27.054346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2416b4-0a88-482e-b4b9-130f0fc5a6bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:27.059224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-1e2416b4-0a88-482e-b4b9-130f0fc5a6bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:37:27.059224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:27.059224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:27.059224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:27.059224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:27.059224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:27.059224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:27.059224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:27.059224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:27.059224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:27.059224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:27.059224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:27.059224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:27.059224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:27.059224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:27.059224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:27.060060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:27.060060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:27.060060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:27.060060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:27.060060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:27.060060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:27.060060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:27.060060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:27.060060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:27.060060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:27.060060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:27.060060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:27.060060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:27.060060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:27.060060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:27.060060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:27.060807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:27.060807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:27.060807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:27.060807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:27.060807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:27.060807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:27.060807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:27.060807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:27.060807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:27.060807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:27.060807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:37:27.060807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:37:27.060807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 37/72] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:37:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:27.062467 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092eb66-7d5a-4604-92ef-930612d03f68 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:27.062467 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092eb66-7d5a-4604-92ef-930612d03f68 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:27.062705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092eb66-7d5a-4604-92ef-930612d03f68 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:27.063999 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 38/73] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:27.064751 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092eb66-7d5a-4604-92ef-930612d03f68 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:27.084998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:27.102337 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:27.142092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4c78c-abc6-4c35-8b5e-e03489289723 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:27.170524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092eb66-7d5a-4604-92ef-930612d03f68 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:27.177202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fbf03-7b0a-41e1-bfe5-ed6b1a22e4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:27.177812 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fbf03-7b0a-41e1-bfe5-ed6b1a22e4dd None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:27.178086 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fbf03-7b0a-41e1-bfe5-ed6b1a22e4dd None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:27.178362 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fbf03-7b0a-41e1-bfe5-ed6b1a22e4dd None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:27.179031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fbf03-7b0a-41e1-bfe5-ed6b1a22e4dd None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:27.179366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fbf03-7b0a-41e1-bfe5-ed6b1a22e4dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:27.180770 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fbf03-7b0a-41e1-bfe5-ed6b1a22e4dd None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:27.188757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 36/74] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:27.189534 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fbf03-7b0a-41e1-bfe5-ed6b1a22e4dd None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:27.204462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d24fb-4e53-4a0a-95b1-9db910d3e214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:27.205070 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d24fb-4e53-4a0a-95b1-9db910d3e214 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:27.205355 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d24fb-4e53-4a0a-95b1-9db910d3e214 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:27.205632 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d24fb-4e53-4a0a-95b1-9db910d3e214 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:27.206304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d24fb-4e53-4a0a-95b1-9db910d3e214 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:27.206619 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d24fb-4e53-4a0a-95b1-9db910d3e214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:27.208044 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d24fb-4e53-4a0a-95b1-9db910d3e214 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:27.216145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 39/75] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:27.221109 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd40a2-427e-4da5-aa40-b6485e22144b None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:27.221109 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd40a2-427e-4da5-aa40-b6485e22144b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:27.221109 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd40a2-427e-4da5-aa40-b6485e22144b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:27.236290 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-effd40a2-427e-4da5-aa40-b6485e22144b None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:27.237516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-effd40a2-427e-4da5-aa40-b6485e22144b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:27.292383 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fbf03-7b0a-41e1-bfe5-ed6b1a22e4dd None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:27.299868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:27.308079 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 40/76] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:27.308921 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd40a2-427e-4da5-aa40-b6485e22144b None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:27.328998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2d225-716e-489a-bafd-8b496b62b5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:27.329643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2d225-716e-489a-bafd-8b496b62b5e9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:27.329933 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2d225-716e-489a-bafd-8b496b62b5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:27.330210 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2d225-716e-489a-bafd-8b496b62b5e9 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:27.330878 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2d225-716e-489a-bafd-8b496b62b5e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:27.331213 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2d225-716e-489a-bafd-8b496b62b5e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:27.332631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2d225-716e-489a-bafd-8b496b62b5e9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:27.335980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:27.345656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 37/77] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:37:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:27.345656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2d225-716e-489a-bafd-8b496b62b5e9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:27.421992 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd40a2-427e-4da5-aa40-b6485e22144b None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:27.455622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2d225-716e-489a-bafd-8b496b62b5e9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:27.458458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a019481-9af5-4f57-bead-850fb501f71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:27.459079 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a019481-9af5-4f57-bead-850fb501f71d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:27.459391 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a019481-9af5-4f57-bead-850fb501f71d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:27.459669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a019481-9af5-4f57-bead-850fb501f71d None admin] PATH_INFO: `/v3/roles/member` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:27.471853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a019481-9af5-4f57-bead-850fb501f71d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:27.478510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-5a019481-9af5-4f57-bead-850fb501f71d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:27.478510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:27.478510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:27.478510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:27.478510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:27.478510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:27.478510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:27.478510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:27.478510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:27.478510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:27.478510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:27.478510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:27.478510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:27.478510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:27.478510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:27.478510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:27.479396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:27.479396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:27.479396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:27.479396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:27.479396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:27.479396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:27.479396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:27.479396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:27.479396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:27.479396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:27.479396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:27.479396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:27.479396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:27.479396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:27.479396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:27.479396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:27.480155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:27.480155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:27.480155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:27.480155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:27.480155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:27.480155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:27.480155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:27.480155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:27.480155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:27.480155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:27.480155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:27.480155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:37:27.480155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 41/78] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:27.486506 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a019481-9af5-4f57-bead-850fb501f71d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:27.494248 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe81ce-ae7f-4ffa-af3e-bacc915924bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:27.494886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe81ce-ae7f-4ffa-af3e-bacc915924bd None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:27.495204 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe81ce-ae7f-4ffa-af3e-bacc915924bd None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:27.495482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe81ce-ae7f-4ffa-af3e-bacc915924bd None admin] PATH_INFO: `/v3/users/6827d81000d9402b8bf49e5bf4aed9cd` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:27.509873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe81ce-ae7f-4ffa-af3e-bacc915924bd None admin] RBAC: Authorizing `identity:get_user(user_id=6827d81000d9402b8bf49e5bf4aed9cd)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:27.511451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe81ce-ae7f-4ffa-af3e-bacc915924bd None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:27.517916 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 38/79] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:27 2022] GET /identity/v3/users/6827d81000d9402b8bf49e5bf4aed9cd => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:27.517916 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe81ce-ae7f-4ffa-af3e-bacc915924bd None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:27.545240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:27.577791 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:27.592629 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a019481-9af5-4f57-bead-850fb501f71d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:27.617577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe81ce-ae7f-4ffa-af3e-bacc915924bd None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:27.626475 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e64f5-bff8-46c8-aa9a-689586b0cf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:27.627126 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e64f5-bff8-46c8-aa9a-689586b0cf66 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:27.627413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e64f5-bff8-46c8-aa9a-689586b0cf66 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:27.627689 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e64f5-bff8-46c8-aa9a-689586b0cf66 None admin] PATH_INFO: `/v3/users/ef040204e84943a7ad9a173f7dd9bb2b` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:27.629070 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e64f5-bff8-46c8-aa9a-689586b0cf66 None admin] RBAC: Authorizing `identity:get_user(user_id=ef040204e84943a7ad9a173f7dd9bb2b)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:27.630526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e64f5-bff8-46c8-aa9a-689586b0cf66 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:27.632440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 42/80] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:27 2022] GET /identity/v3/users/ef040204e84943a7ad9a173f7dd9bb2b => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:27.633214 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e64f5-bff8-46c8-aa9a-689586b0cf66 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:27.651396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f40e88-c096-44a0-9256-b7d6a0be75a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:27.652029 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f40e88-c096-44a0-9256-b7d6a0be75a9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:27.652308 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f40e88-c096-44a0-9256-b7d6a0be75a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:27.652587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f40e88-c096-44a0-9256-b7d6a0be75a9 None admin] PATH_INFO: `/v3/users/6827d81000d9402b8bf49e5bf4aed9cd` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:27.654002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f40e88-c096-44a0-9256-b7d6a0be75a9 None admin] RBAC: Authorizing `identity:get_user(user_id=6827d81000d9402b8bf49e5bf4aed9cd)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:27.656037 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f40e88-c096-44a0-9256-b7d6a0be75a9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:27.657610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 39/81] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:27 2022] GET /identity/v3/users/6827d81000d9402b8bf49e5bf4aed9cd => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:27.659442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f40e88-c096-44a0-9256-b7d6a0be75a9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:27.732810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e64f5-bff8-46c8-aa9a-689586b0cf66 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:27.748115 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:27.754447 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f40e88-c096-44a0-9256-b7d6a0be75a9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:27.766116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56ce0b-7632-4514-8d76-53beb57abc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:27.766725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56ce0b-7632-4514-8d76-53beb57abc78 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:27.767007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56ce0b-7632-4514-8d76-53beb57abc78 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:27.767311 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56ce0b-7632-4514-8d76-53beb57abc78 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:27.768013 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56ce0b-7632-4514-8d76-53beb57abc78 None admin] RBAC: Adding query filter params (name=member) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:27.768320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56ce0b-7632-4514-8d76-53beb57abc78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:27.769738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56ce0b-7632-4514-8d76-53beb57abc78 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:27.777687 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 43/82] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:27.778469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56ce0b-7632-4514-8d76-53beb57abc78 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:27.789363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14872cd7-983f-4862-8488-315c11c8ea56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:27.789972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14872cd7-983f-4862-8488-315c11c8ea56 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:27.790254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14872cd7-983f-4862-8488-315c11c8ea56 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:27.790526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14872cd7-983f-4862-8488-315c11c8ea56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:27.797223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:27.856552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14872cd7-983f-4862-8488-315c11c8ea56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:27.858011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14872cd7-983f-4862-8488-315c11c8ea56 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:27.877063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56ce0b-7632-4514-8d76-53beb57abc78 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:27.911390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069df4e-4f5c-4371-aa21-4f8e1c2c5fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:27.912014 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069df4e-4f5c-4371-aa21-4f8e1c2c5fd7 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:27.912295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069df4e-4f5c-4371-aa21-4f8e1c2c5fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:27.912569 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069df4e-4f5c-4371-aa21-4f8e1c2c5fd7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:27.913732 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069df4e-4f5c-4371-aa21-4f8e1c2c5fd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:27.915203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069df4e-4f5c-4371-aa21-4f8e1c2c5fd7 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:27.916763 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 44/83] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:27.917523 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069df4e-4f5c-4371-aa21-4f8e1c2c5fd7 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:27.950094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:27.957900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 40/84] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:27.958740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14872cd7-983f-4862-8488-315c11c8ea56 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:28.000197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:28.015964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069df4e-4f5c-4371-aa21-4f8e1c2c5fd7 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:28.050327 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cb471-a8f4-4678-bcd8-210fd2d2fb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:28.050856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cb471-a8f4-4678-bcd8-210fd2d2fb51 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:28.051158 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cb471-a8f4-4678-bcd8-210fd2d2fb51 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:28.051436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cb471-a8f4-4678-bcd8-210fd2d2fb51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:28.056270 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14872cd7-983f-4862-8488-315c11c8ea56 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:28.090704 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebeba3-7e52-419c-9c25-6c2504a46f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:28.091340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebeba3-7e52-419c-9c25-6c2504a46f00 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:28.091621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebeba3-7e52-419c-9c25-6c2504a46f00 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:28.091891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebeba3-7e52-419c-9c25-6c2504a46f00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:28.116280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3cb471-a8f4-4678-bcd8-210fd2d2fb51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:28.117747 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3cb471-a8f4-4678-bcd8-210fd2d2fb51 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:28.153141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:28.161112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebeba3-7e52-419c-9c25-6c2504a46f00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:28.162599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebeba3-7e52-419c-9c25-6c2504a46f00 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:28.198434 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:28.214913 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 45/85] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:28.215753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cb471-a8f4-4678-bcd8-210fd2d2fb51 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:28.275198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 41/86] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:28.276040 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebeba3-7e52-419c-9c25-6c2504a46f00 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:28.321966 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cb471-a8f4-4678-bcd8-210fd2d2fb51 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:28.355500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856b56f-7e3d-4459-aa10-4e32c3802cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:28.356127 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856b56f-7e3d-4459-aa10-4e32c3802cb2 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:28.356416 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856b56f-7e3d-4459-aa10-4e32c3802cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:28.356693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856b56f-7e3d-4459-aa10-4e32c3802cb2 None admin] PATH_INFO: `/v3/users/6827d81000d9402b8bf49e5bf4aed9cd` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:28.357832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856b56f-7e3d-4459-aa10-4e32c3802cb2 None admin] RBAC: Authorizing `identity:get_user(user_id=6827d81000d9402b8bf49e5bf4aed9cd)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:28.359819 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856b56f-7e3d-4459-aa10-4e32c3802cb2 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:28.361405 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 46/87] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:28 2022] GET /identity/v3/users/6827d81000d9402b8bf49e5bf4aed9cd => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:28.364176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856b56f-7e3d-4459-aa10-4e32c3802cb2 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:28.367938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:28.377717 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebeba3-7e52-419c-9c25-6c2504a46f00 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:28.410426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5fd56-d0ed-4595-ada5-99957ed759b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:28.411035 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5fd56-d0ed-4595-ada5-99957ed759b8 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:28.411362 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5fd56-d0ed-4595-ada5-99957ed759b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:28.411822 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5fd56-d0ed-4595-ada5-99957ed759b8 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:28.412704 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5fd56-d0ed-4595-ada5-99957ed759b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:28.414746 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5fd56-d0ed-4595-ada5-99957ed759b8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:28.415891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:28.462668 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856b56f-7e3d-4459-aa10-4e32c3802cb2 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:28.495189 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c068c-527b-4ed9-af5b-8f3695a8dfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:28.495802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c068c-527b-4ed9-af5b-8f3695a8dfb0 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:28.496079 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c068c-527b-4ed9-af5b-8f3695a8dfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:28.496349 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c068c-527b-4ed9-af5b-8f3695a8dfb0 None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:28.497452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c068c-527b-4ed9-af5b-8f3695a8dfb0 None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:28.498942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c068c-527b-4ed9-af5b-8f3695a8dfb0 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:28.501123 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 47/88] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:28 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:28.501874 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c068c-527b-4ed9-af5b-8f3695a8dfb0 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:28.526197 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 42/89] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:37:28 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:37:28.536231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5fd56-d0ed-4595-ada5-99957ed759b8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:28.612066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c068c-527b-4ed9-af5b-8f3695a8dfb0 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:28.637875 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:28.650367 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5fd56-d0ed-4595-ada5-99957ed759b8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:28.652724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8695d1-3e42-4369-97fb-cfb272fe0a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:28.653432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8695d1-3e42-4369-97fb-cfb272fe0a77 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:28.653912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8695d1-3e42-4369-97fb-cfb272fe0a77 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:28.654193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8695d1-3e42-4369-97fb-cfb272fe0a77 None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:28.655994 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8695d1-3e42-4369-97fb-cfb272fe0a77 None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:28.657597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8695d1-3e42-4369-97fb-cfb272fe0a77 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:28.659424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 48/90] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:28 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:28.665409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8695d1-3e42-4369-97fb-cfb272fe0a77 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:28.697243 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763a10b-bbac-4bc3-a34d-189c20afacd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:28.698103 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763a10b-bbac-4bc3-a34d-189c20afacd8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:28.698401 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763a10b-bbac-4bc3-a34d-189c20afacd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:28.699061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763a10b-bbac-4bc3-a34d-189c20afacd8 None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:28.700283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763a10b-bbac-4bc3-a34d-189c20afacd8 None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:28.701819 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763a10b-bbac-4bc3-a34d-189c20afacd8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:28.703771 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 43/91] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:28 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:28.704364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763a10b-bbac-4bc3-a34d-189c20afacd8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:28.726059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:28.791748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8695d1-3e42-4369-97fb-cfb272fe0a77 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:28.825796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763a10b-bbac-4bc3-a34d-189c20afacd8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:28.842405 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff0180-c9ea-44ae-ab31-ce969354aeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:28.843579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff0180-c9ea-44ae-ab31-ce969354aeee None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:28.843864 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff0180-c9ea-44ae-ab31-ce969354aeee None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:28.844526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff0180-c9ea-44ae-ab31-ce969354aeee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:28.868793 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32564ffe-4d0a-406c-b378-55b407f01129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:28.869380 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32564ffe-4d0a-406c-b378-55b407f01129 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:28.869677 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32564ffe-4d0a-406c-b378-55b407f01129 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:28.869957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32564ffe-4d0a-406c-b378-55b407f01129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:28.870709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32564ffe-4d0a-406c-b378-55b407f01129 None admin] RBAC: Adding query filter params (role.id=7c4463788f534454b6d8319dbc4d2afc, scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4, user.id=6827d81000d9402b8bf49e5bf4aed9cd) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:28.871025 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32564ffe-4d0a-406c-b378-55b407f01129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:28.872528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32564ffe-4d0a-406c-b378-55b407f01129 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:28.880969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 44/92] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:28 2022] GET /identity/v3/role_assignments?user.id=6827d81000d9402b8bf49e5bf4aed9cd&scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4&role.id=7c4463788f534454b6d8319dbc4d2afc => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:28.882903 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32564ffe-4d0a-406c-b378-55b407f01129 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:28.945832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:28.967145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff0180-c9ea-44ae-ab31-ce969354aeee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:28.974127 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff0180-c9ea-44ae-ab31-ce969354aeee None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:29.057813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:29.081303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32564ffe-4d0a-406c-b378-55b407f01129 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:29.118768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fac44-dc1c-4704-85fe-a57a899e8a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:29.119619 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fac44-dc1c-4704-85fe-a57a899e8a92 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:29.119947 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fac44-dc1c-4704-85fe-a57a899e8a92 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:29.120344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fac44-dc1c-4704-85fe-a57a899e8a92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:29.121252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 49/93] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:29.121528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00fac44-dc1c-4704-85fe-a57a899e8a92 None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4, user.id=ef040204e84943a7ad9a173f7dd9bb2b) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:29.121948 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00fac44-dc1c-4704-85fe-a57a899e8a92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:29.122591 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff0180-c9ea-44ae-ab31-ce969354aeee None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:29.123641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00fac44-dc1c-4704-85fe-a57a899e8a92 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:29.130031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 45/94] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:28 2022] GET /identity/v3/role_assignments?user.id=ef040204e84943a7ad9a173f7dd9bb2b&scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:29.130871 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fac44-dc1c-4704-85fe-a57a899e8a92 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:29.247962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fac44-dc1c-4704-85fe-a57a899e8a92 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:29.291696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ce8de-07a3-413e-8424-9e10e7b24ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:29.293136 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ce8de-07a3-413e-8424-9e10e7b24ca5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:29.293627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ce8de-07a3-413e-8424-9e10e7b24ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:29.294093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ce8de-07a3-413e-8424-9e10e7b24ca5 None admin] PATH_INFO: `/v3/projects/b7a794852e15470b8aa1f62923197266` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:29.295484 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ce8de-07a3-413e-8424-9e10e7b24ca5 None admin] RBAC: Authorizing `identity:get_project(project_id=b7a794852e15470b8aa1f62923197266)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:29.297260 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ce8de-07a3-413e-8424-9e10e7b24ca5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:29.299278 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 46/95] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:29 2022] GET /identity/v3/projects/b7a794852e15470b8aa1f62923197266 => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:29.301371 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:29.306043 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ce8de-07a3-413e-8424-9e10e7b24ca5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:29.323614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:29.337782 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff0180-c9ea-44ae-ab31-ce969354aeee None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:29.421777 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ce8de-07a3-413e-8424-9e10e7b24ca5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:29.430616 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48d83c-c9a5-4bd6-b6fc-eaef0ebe7afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:29.431205 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48d83c-c9a5-4bd6-b6fc-eaef0ebe7afc None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:29.431490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48d83c-c9a5-4bd6-b6fc-eaef0ebe7afc None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:29.431763 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48d83c-c9a5-4bd6-b6fc-eaef0ebe7afc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:29.432508 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48d83c-c9a5-4bd6-b6fc-eaef0ebe7afc None admin] RBAC: Adding query filter params (role.id=6fd5089acea448219d4c0a9a9424a3af, scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4, user.id=6827d81000d9402b8bf49e5bf4aed9cd) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:29.432814 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48d83c-c9a5-4bd6-b6fc-eaef0ebe7afc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:29.434266 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48d83c-c9a5-4bd6-b6fc-eaef0ebe7afc None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:29.452782 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 50/96] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:29 2022] GET /identity/v3/role_assignments?user.id=6827d81000d9402b8bf49e5bf4aed9cd&scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4&role.id=6fd5089acea448219d4c0a9a9424a3af => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:29.470950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729bb42-0952-4c86-9471-7358dd1ad9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:29.472026 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729bb42-0952-4c86-9471-7358dd1ad9c1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:29.472508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729bb42-0952-4c86-9471-7358dd1ad9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:29.473022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729bb42-0952-4c86-9471-7358dd1ad9c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:29.473964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729bb42-0952-4c86-9471-7358dd1ad9c1 None admin] RBAC: Adding query filter params (role.id=7c4463788f534454b6d8319dbc4d2afc, scope.project.id=b7a794852e15470b8aa1f62923197266, user.id=6827d81000d9402b8bf49e5bf4aed9cd) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:29.474491 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729bb42-0952-4c86-9471-7358dd1ad9c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:29.476182 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0729bb42-0952-4c86-9471-7358dd1ad9c1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:29.482756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 47/97] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:29 2022] GET /identity/v3/role_assignments?user.id=6827d81000d9402b8bf49e5bf4aed9cd&scope.project.id=b7a794852e15470b8aa1f62923197266&role.id=7c4463788f534454b6d8319dbc4d2afc => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:31.014043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0483f2-57df-4918-8282-a30d1cafb762 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:31.015089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0483f2-57df-4918-8282-a30d1cafb762 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:31.015293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0483f2-57df-4918-8282-a30d1cafb762 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:31.016503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 51/98] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:31.032993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464a775-5833-4404-9dc2-a46d420753f7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:31.033436 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464a775-5833-4404-9dc2-a46d420753f7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:31.033640 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464a775-5833-4404-9dc2-a46d420753f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:31.057645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0464a775-5833-4404-9dc2-a46d420753f7 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:31.058863 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0464a775-5833-4404-9dc2-a46d420753f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:31.132293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 48/99] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:31.362696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417766f-b370-4ef6-8c28-9f68d08a1cef None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:31.363007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417766f-b370-4ef6-8c28-9f68d08a1cef None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:31.363291 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417766f-b370-4ef6-8c28-9f68d08a1cef None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:31.364867 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 52/100] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:31.370147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2dd4b-d27c-4d94-bf6c-576d72787bc7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:31.370443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2dd4b-d27c-4d94-bf6c-576d72787bc7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:31.370729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2dd4b-d27c-4d94-bf6c-576d72787bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:31.389917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-79c2dd4b-d27c-4d94-bf6c-576d72787bc7 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:31.391130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-79c2dd4b-d27c-4d94-bf6c-576d72787bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:31.397266 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:31.481232 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 49/101] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:31.488876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417766f-b370-4ef6-8c28-9f68d08a1cef None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:31.656289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417766f-b370-4ef6-8c28-9f68d08a1cef None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:31.699784 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7e9d0-e721-4936-a70a-f9b8a8f97ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:31.700763 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7e9d0-e721-4936-a70a-f9b8a8f97ce9 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:31.701253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7e9d0-e721-4936-a70a-f9b8a8f97ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:31.701713 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7e9d0-e721-4936-a70a-f9b8a8f97ce9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:31.712369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7e9d0-e721-4936-a70a-f9b8a8f97ce9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:31.717363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-7ab7e9d0-e721-4936-a70a-f9b8a8f97ce9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:37:31.717363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:31.717363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:31.717363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:31.717363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:31.717363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:31.717363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:31.717363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:31.717363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:31.717363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:31.717363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:31.717363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:31.717363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:31.717363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:31.717363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:31.717363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:31.718710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:31.718710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:31.718710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:31.718710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:31.718710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:31.718710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:31.718710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:31.718710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:31.718710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:31.718710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:31.718710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:31.718710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:31.718710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:31.718710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:31.718710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:31.718710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:31.719490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:31.719490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:31.719490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:31.719490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:31.719490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:31.719490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:31.719490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:31.719490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:31.719490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:31.719490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:31.719490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:37:31.719490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:37:31.726355 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 53/102] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:31.727923 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2dd4b-d27c-4d94-bf6c-576d72787bc7 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:31.851407 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2dd4b-d27c-4d94-bf6c-576d72787bc7 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:31.898029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8127f-151a-4167-af01-9afedcc49d35 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:31.898340 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8127f-151a-4167-af01-9afedcc49d35 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:31.898594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8127f-151a-4167-af01-9afedcc49d35 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:31.899384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a1105-ed72-46b7-bc52-301c5fb82a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:31.900041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 54/103] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:37:31.900493 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a1105-ed72-46b7-bc52-301c5fb82a4c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:31.900962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a1105-ed72-46b7-bc52-301c5fb82a4c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:31.901423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a1105-ed72-46b7-bc52-301c5fb82a4c None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:31.902288 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a1105-ed72-46b7-bc52-301c5fb82a4c None admin] RBAC: Adding query filter params (name=reader) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:31.902781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a1105-ed72-46b7-bc52-301c5fb82a4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:31.904750 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a1105-ed72-46b7-bc52-301c5fb82a4c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:31.912306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d1b92-1f80-459e-ac27-eac4a66e9fe8 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:31.912529 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d1b92-1f80-459e-ac27-eac4a66e9fe8 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:31.912808 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d1b92-1f80-459e-ac27-eac4a66e9fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:31.914378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:31.920137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 50/104] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:31.926436 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a1105-ed72-46b7-bc52-301c5fb82a4c None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:31.934318 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-363d1b92-1f80-459e-ac27-eac4a66e9fe8 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:31.936640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-363d1b92-1f80-459e-ac27-eac4a66e9fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:31.945396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:32.068848 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 55/105] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:37:32.069790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d1b92-1f80-459e-ac27-eac4a66e9fe8 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:32.141054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a1105-ed72-46b7-bc52-301c5fb82a4c None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:32.197685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b227558-f60e-4f1e-93fa-abe490a0fd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:32.198288 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b227558-f60e-4f1e-93fa-abe490a0fd99 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:32.198547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b227558-f60e-4f1e-93fa-abe490a0fd99 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:32.198827 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b227558-f60e-4f1e-93fa-abe490a0fd99 None admin] PATH_INFO: `/v3/users/ce5c8b94724b444f90b8cf10caac2c1a` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:32.215092 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d1b92-1f80-459e-ac27-eac4a66e9fe8 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:32.222603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b227558-f60e-4f1e-93fa-abe490a0fd99 None admin] RBAC: Authorizing `identity:get_user(user_id=ce5c8b94724b444f90b8cf10caac2c1a)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:32.224160 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b227558-f60e-4f1e-93fa-abe490a0fd99 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:32.231595 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 51/106] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:31 2022] GET /identity/v3/users/ce5c8b94724b444f90b8cf10caac2c1a => generated 289 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:32.245836 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b227558-f60e-4f1e-93fa-abe490a0fd99 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:32.296328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59dbe0-2753-41fc-9fc1-34ea70b8fdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:32.296970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59dbe0-2753-41fc-9fc1-34ea70b8fdff None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:32.297253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59dbe0-2753-41fc-9fc1-34ea70b8fdff None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:32.297530 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59dbe0-2753-41fc-9fc1-34ea70b8fdff None admin] PATH_INFO: `/v3/domains/default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:32.298744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59dbe0-2753-41fc-9fc1-34ea70b8fdff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:32.303369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59dbe0-2753-41fc-9fc1-34ea70b8fdff None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:32.305520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 56/107] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:32.306694 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4578f-672a-4664-b5b1-ac99687fa5a8 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:32.306978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4578f-672a-4664-b5b1-ac99687fa5a8 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:32.307283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4578f-672a-4664-b5b1-ac99687fa5a8 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:32.314635 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 57/108] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:32.315384 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4578f-672a-4664-b5b1-ac99687fa5a8 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:32.334158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:32.338627 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:32.374647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b227558-f60e-4f1e-93fa-abe490a0fd99 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:32.411124 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb72a7-3b4e-4d7a-91de-87664f8c487b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:32.411768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb72a7-3b4e-4d7a-91de-87664f8c487b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:32.412244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb72a7-3b4e-4d7a-91de-87664f8c487b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:32.412667 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb72a7-3b4e-4d7a-91de-87664f8c487b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:32.449978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4578f-672a-4664-b5b1-ac99687fa5a8 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:32.487676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d1edf-c091-4c62-9846-aebbec002a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:32.488286 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d1edf-c091-4c62-9846-aebbec002a65 None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:32.488566 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d1edf-c091-4c62-9846-aebbec002a65 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:32.488845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d1edf-c091-4c62-9846-aebbec002a65 None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:32.489505 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d1edf-c091-4c62-9846-aebbec002a65 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:32.490919 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb72a7-3b4e-4d7a-91de-87664f8c487b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:32.491025 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d1edf-c091-4c62-9846-aebbec002a65 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:32.492662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb72a7-3b4e-4d7a-91de-87664f8c487b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:32.550886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:32.579873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 58/109] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 30 14:37:32 2022] POST /identity/v3/projects => generated 302 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:37:32.579873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675a5b7-7874-45bd-a683-352247314c27 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:32.579873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675a5b7-7874-45bd-a683-352247314c27 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:32.579873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675a5b7-7874-45bd-a683-352247314c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:32.594607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-0675a5b7-7874-45bd-a683-352247314c27 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:32.595911 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0675a5b7-7874-45bd-a683-352247314c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:32.609334 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 52/110] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:32.610680 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f9514-831d-4125-8847-4a5e9a290caf None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:32.610978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f9514-831d-4125-8847-4a5e9a290caf None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:32.611330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f9514-831d-4125-8847-4a5e9a290caf None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:32.612868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 53/111] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:32.613608 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f9514-831d-4125-8847-4a5e9a290caf None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:32.679717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:32.684282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 59/112] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:32.685320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675a5b7-7874-45bd-a683-352247314c27 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:32.767639 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f9514-831d-4125-8847-4a5e9a290caf None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:32.807219 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675a5b7-7874-45bd-a683-352247314c27 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:32.810534 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c219beb-39f0-4878-80f8-558b9b9997a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:32.811330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c219beb-39f0-4878-80f8-558b9b9997a5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:32.812071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c219beb-39f0-4878-80f8-558b9b9997a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:32.812346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c219beb-39f0-4878-80f8-558b9b9997a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:32.827159 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c219beb-39f0-4878-80f8-558b9b9997a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:32.832128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-1c219beb-39f0-4878-80f8-558b9b9997a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:32.832128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:32.832128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:32.832128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:32.832128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:32.832128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:32.832128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:32.832128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:32.832128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:32.832128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:32.832128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:32.832128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:32.832128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:32.832128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:32.832128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:32.832128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:32.832952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:32.832952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:32.832952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:32.832952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:32.832952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:32.832952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:32.832952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:32.832952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:32.832952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:32.832952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:32.832952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:32.832952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:32.832952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:32.832952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:32.832952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:32.832952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:32.833732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:32.833732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:32.833732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:32.833732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:32.833732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:32.833732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:32.833732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:32.833732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:32.833732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:32.833732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:32.833732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:32.833732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:32.833732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 54/113] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:37:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:32.837047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c6008-f488-477e-aaeb-6da936471590 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:32.837047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c6008-f488-477e-aaeb-6da936471590 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:32.837047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c6008-f488-477e-aaeb-6da936471590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:32.853777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceed2a1-62b6-424f-bb78-74bb1ca376b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:32.854323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-511c6008-f488-477e-aaeb-6da936471590 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:32.854428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceed2a1-62b6-424f-bb78-74bb1ca376b1 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:32.854684 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceed2a1-62b6-424f-bb78-74bb1ca376b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:32.855009 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceed2a1-62b6-424f-bb78-74bb1ca376b1 None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:32.855566 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-511c6008-f488-477e-aaeb-6da936471590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:32.856332 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceed2a1-62b6-424f-bb78-74bb1ca376b1 None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:32.858832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceed2a1-62b6-424f-bb78-74bb1ca376b1 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:32.861453 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 60/114] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:32 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:32.862757 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ceaf6a-0d85-43f9-bdda-a8db44206f60 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:32.863245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ceaf6a-0d85-43f9-bdda-a8db44206f60 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:32.863695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ceaf6a-0d85-43f9-bdda-a8db44206f60 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:32.865496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 61/115] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:32.866378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ceaf6a-0d85-43f9-bdda-a8db44206f60 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:32.871813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:32.931953 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 55/116] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:32.932824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c6008-f488-477e-aaeb-6da936471590 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:32.943012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:32.996600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ceaf6a-0d85-43f9-bdda-a8db44206f60 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:33.033446 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2faca-a3ec-4755-a8e4-40d899b3050f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:33.034390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2faca-a3ec-4755-a8e4-40d899b3050f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:33.034677 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2faca-a3ec-4755-a8e4-40d899b3050f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:33.034954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2faca-a3ec-4755-a8e4-40d899b3050f None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:33.036150 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2faca-a3ec-4755-a8e4-40d899b3050f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:33.036514 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2faca-a3ec-4755-a8e4-40d899b3050f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:33.038538 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2faca-a3ec-4755-a8e4-40d899b3050f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:33.054892 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 62/117] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:33.054892 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acae74-f362-4edc-8f72-5991f4c13392 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:33.054892 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acae74-f362-4edc-8f72-5991f4c13392 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:33.054892 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acae74-f362-4edc-8f72-5991f4c13392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:33.062997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c6008-f488-477e-aaeb-6da936471590 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:33.075587 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e7acae74-f362-4edc-8f72-5991f4c13392 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:33.077178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e7acae74-f362-4edc-8f72-5991f4c13392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:33.107078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92bbff-0627-4dfe-af50-a6606bdb1aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:33.107995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92bbff-0627-4dfe-af50-a6606bdb1aeb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:33.108906 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92bbff-0627-4dfe-af50-a6606bdb1aeb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:33.109179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92bbff-0627-4dfe-af50-a6606bdb1aeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:33.110240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92bbff-0627-4dfe-af50-a6606bdb1aeb None admin] RBAC: Adding query filter params (role.id=486acd6fe1174148b9461c56c9d371aa, scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4, user.id=ce5c8b94724b444f90b8cf10caac2c1a) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:33.110620 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92bbff-0627-4dfe-af50-a6606bdb1aeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:33.112877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92bbff-0627-4dfe-af50-a6606bdb1aeb None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:33.119988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 56/118] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:32 2022] GET /identity/v3/role_assignments?user.id=ce5c8b94724b444f90b8cf10caac2c1a&scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4&role.id=486acd6fe1174148b9461c56c9d371aa => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:33.121786 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92bbff-0627-4dfe-af50-a6606bdb1aeb None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:33.144157 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:33.160879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 63/119] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:33.162222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acae74-f362-4edc-8f72-5991f4c13392 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:33.224940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:33.277891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92bbff-0627-4dfe-af50-a6606bdb1aeb None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:33.301359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acae74-f362-4edc-8f72-5991f4c13392 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:33.320966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb52721-c027-4226-ad7b-efe504bf852f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:33.321625 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb52721-c027-4226-ad7b-efe504bf852f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:33.321911 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb52721-c027-4226-ad7b-efe504bf852f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:33.322187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb52721-c027-4226-ad7b-efe504bf852f None admin] PATH_INFO: `/v3/roles/member` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:33.336396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb52721-c027-4226-ad7b-efe504bf852f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:33.343185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-eeb52721-c027-4226-ad7b-efe504bf852f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:33.343185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:33.343185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:33.343185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:33.343185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:33.343185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:33.343185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:33.343185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:33.343185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:33.343185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:33.343185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:33.343185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:33.343185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:33.343185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:33.343185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:33.343185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:33.344059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:33.344059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:33.344059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:33.344059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:33.344059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:33.344059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:33.344059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:33.344059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:33.344059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:33.344059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:33.344059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:33.344059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:33.344059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:33.344059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:33.344059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:33.344059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:33.344821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:33.344821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:33.344821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:33.344821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:33.344821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:33.344821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:33.344821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:33.344821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:33.344821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:33.344821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:33.344821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:33.344821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:33.346055 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 57/120] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:33.346055 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb52721-c027-4226-ad7b-efe504bf852f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:33.391389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0095336-8be3-43d1-b275-b81bc89955e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:33.392031 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0095336-8be3-43d1-b275-b81bc89955e4 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:33.392488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0095336-8be3-43d1-b275-b81bc89955e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:33.392907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0095336-8be3-43d1-b275-b81bc89955e4 None admin] PATH_INFO: `/v3/users/ef040204e84943a7ad9a173f7dd9bb2b` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:33.394190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0095336-8be3-43d1-b275-b81bc89955e4 None admin] RBAC: Authorizing `identity:get_user(user_id=ef040204e84943a7ad9a173f7dd9bb2b)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:33.395835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0095336-8be3-43d1-b275-b81bc89955e4 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:33.397886 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 64/121] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:33 2022] GET /identity/v3/users/ef040204e84943a7ad9a173f7dd9bb2b => generated 247 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:33.398829 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0095336-8be3-43d1-b275-b81bc89955e4 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:33.432393 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:33.497139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb52721-c027-4226-ad7b-efe504bf852f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:33.508100 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0095336-8be3-43d1-b275-b81bc89955e4 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:33.532931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:33.552815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc6434-3dfa-4c72-8758-60089b5c2d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:33.553406 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc6434-3dfa-4c72-8758-60089b5c2d95 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:33.553686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc6434-3dfa-4c72-8758-60089b5c2d95 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:33.553959 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc6434-3dfa-4c72-8758-60089b5c2d95 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:33.554639 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc6434-3dfa-4c72-8758-60089b5c2d95 None admin] RBAC: Adding query filter params (name=member) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:33.554944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc6434-3dfa-4c72-8758-60089b5c2d95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:33.556460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc6434-3dfa-4c72-8758-60089b5c2d95 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:33.565193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee362f9-b230-42dc-944b-d70da52f359f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:33.566075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee362f9-b230-42dc-944b-d70da52f359f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:33.566537 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee362f9-b230-42dc-944b-d70da52f359f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:33.566990 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee362f9-b230-42dc-944b-d70da52f359f None admin] PATH_INFO: `/v3/roles/member` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:33.567628 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 65/122] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:33.568423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc6434-3dfa-4c72-8758-60089b5c2d95 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:33.579079 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee362f9-b230-42dc-944b-d70da52f359f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:33.584721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-4ee362f9-b230-42dc-944b-d70da52f359f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:33.584721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:33.584721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:33.584721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:33.584721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:33.584721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:33.584721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:33.584721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:33.584721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:33.584721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:33.584721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:33.584721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:33.584721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:33.584721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:33.584721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:33.584721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:33.585547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:33.585547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:33.585547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:33.585547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:33.585547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:33.585547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:33.585547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:33.585547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:33.585547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:33.585547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:33.585547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:33.585547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:33.585547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:33.585547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:33.585547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:33.585547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:33.586295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:33.586295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:33.586295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:33.586295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:33.586295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:33.586295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:33.586295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:33.586295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:33.586295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:33.586295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:33.586295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:33.586295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:33.588747 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 58/123] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:33.589721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee362f9-b230-42dc-944b-d70da52f359f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:33.688464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc6434-3dfa-4c72-8758-60089b5c2d95 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:33.705121 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee362f9-b230-42dc-944b-d70da52f359f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:33.725304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a2314-9cda-41f1-8840-da155827fb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:33.726190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a2314-9cda-41f1-8840-da155827fb9e None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:33.726655 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a2314-9cda-41f1-8840-da155827fb9e None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:33.727127 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a2314-9cda-41f1-8840-da155827fb9e None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:33.728474 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a2314-9cda-41f1-8840-da155827fb9e None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:33.730112 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a2314-9cda-41f1-8840-da155827fb9e None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:33.732027 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 66/124] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:33 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:33.733003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a2314-9cda-41f1-8840-da155827fb9e None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:33.743724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:33.751323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f652-704e-4dd8-a5a7-7818f15c125f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:33.752410 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f652-704e-4dd8-a5a7-7818f15c125f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:33.761592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f652-704e-4dd8-a5a7-7818f15c125f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:33.761592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372f652-704e-4dd8-a5a7-7818f15c125f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:33.769097 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372f652-704e-4dd8-a5a7-7818f15c125f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:33.775570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-f372f652-704e-4dd8-a5a7-7818f15c125f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:37:33.775570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:33.775570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:33.775570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:33.775570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:33.775570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:33.775570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:33.775570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:33.775570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:33.775570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:33.775570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:33.775570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:33.775570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:33.775570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:33.775570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:33.775570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:33.776541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:33.776541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:33.776541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:33.776541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:33.776541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:33.776541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:33.776541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:33.776541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:33.776541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:33.776541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:33.776541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:33.776541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:33.776541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:33.776541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:33.776541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:33.776541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:33.777389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:33.777389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:33.777389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:33.777389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:33.777389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:33.777389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:33.777389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:33.777389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:33.777389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:33.777389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:33.777389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:37:33.777389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:33.777389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 59/125] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:37:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:33.778010 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f652-704e-4dd8-a5a7-7818f15c125f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:33.804241 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:33.850126 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a2314-9cda-41f1-8840-da155827fb9e None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:33.887342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103a84b-5716-4d8d-b9fc-c7bb9f488fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:33.888210 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103a84b-5716-4d8d-b9fc-c7bb9f488fd0 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:33.888634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103a84b-5716-4d8d-b9fc-c7bb9f488fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:33.889088 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103a84b-5716-4d8d-b9fc-c7bb9f488fd0 None admin] PATH_INFO: `/v3/users/6827d81000d9402b8bf49e5bf4aed9cd` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:33.890914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103a84b-5716-4d8d-b9fc-c7bb9f488fd0 None admin] RBAC: Authorizing `identity:get_user(user_id=6827d81000d9402b8bf49e5bf4aed9cd)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:33.891991 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372f652-704e-4dd8-a5a7-7818f15c125f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:33.892706 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103a84b-5716-4d8d-b9fc-c7bb9f488fd0 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:33.894708 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 67/126] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:33 2022] GET /identity/v3/users/6827d81000d9402b8bf49e5bf4aed9cd => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:33.895611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103a84b-5716-4d8d-b9fc-c7bb9f488fd0 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:33.928703 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f673672-c142-4516-95c5-603de3949d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:33.929538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f673672-c142-4516-95c5-603de3949d76 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:33.929966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f673672-c142-4516-95c5-603de3949d76 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:33.930481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f673672-c142-4516-95c5-603de3949d76 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:33.931518 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f673672-c142-4516-95c5-603de3949d76 None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:33.931960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f673672-c142-4516-95c5-603de3949d76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:33.933504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f673672-c142-4516-95c5-603de3949d76 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:33.943224 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 60/127] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:33.944309 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f673672-c142-4516-95c5-603de3949d76 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:34.001085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:34.005400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103a84b-5716-4d8d-b9fc-c7bb9f488fd0 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:34.023772 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:34.043232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab1f85b-e50f-4cf3-a49d-beb8ba1d3f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:34.044071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1f85b-e50f-4cf3-a49d-beb8ba1d3f69 None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:34.044567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1f85b-e50f-4cf3-a49d-beb8ba1d3f69 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:34.045009 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1f85b-e50f-4cf3-a49d-beb8ba1d3f69 None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4/users/ef040204e84943a7ad9a173f7dd9bb2b/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:34.047436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab1f85b-e50f-4cf3-a49d-beb8ba1d3f69 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc07ff985d9a458a94b1aefcf6147ae4, user_id=ef040204e84943a7ad9a173f7dd9bb2b, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:34.049020 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab1f85b-e50f-4cf3-a49d-beb8ba1d3f69 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:34.049134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f673672-c142-4516-95c5-603de3949d76 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:34.085190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b6236-0db6-4c03-8c11-972cdc1411cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:34.085733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b6236-0db6-4c03-8c11-972cdc1411cd None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:34.086077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b6236-0db6-4c03-8c11-972cdc1411cd None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:34.086355 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b6236-0db6-4c03-8c11-972cdc1411cd None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:34.087079 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b6236-0db6-4c03-8c11-972cdc1411cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:34.087418 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b6236-0db6-4c03-8c11-972cdc1411cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:34.088872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b6236-0db6-4c03-8c11-972cdc1411cd None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:34.097210 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 61/128] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:37:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:34.098142 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b6236-0db6-4c03-8c11-972cdc1411cd None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:34.129074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 68/129] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:37:33 2022] PUT /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4/users/ef040204e84943a7ad9a173f7dd9bb2b/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:37:34.134600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab1f85b-e50f-4cf3-a49d-beb8ba1d3f69 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:34.229600 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b6236-0db6-4c03-8c11-972cdc1411cd None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:34.255524 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:34.277257 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373a1cd-b8ac-49ee-acba-a8df39af57ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:34.278526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373a1cd-b8ac-49ee-acba-a8df39af57ef None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:34.278990 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373a1cd-b8ac-49ee-acba-a8df39af57ef None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:34.279449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373a1cd-b8ac-49ee-acba-a8df39af57ef None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:34.281063 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373a1cd-b8ac-49ee-acba-a8df39af57ef None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:34.282745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373a1cd-b8ac-49ee-acba-a8df39af57ef None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:34.284887 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 62/130] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:34 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:34.286171 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373a1cd-b8ac-49ee-acba-a8df39af57ef None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:34.363591 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab1f85b-e50f-4cf3-a49d-beb8ba1d3f69 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:34.415180 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736087db-01ed-4f1b-9bd8-a80283bd67cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:34.415826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736087db-01ed-4f1b-9bd8-a80283bd67cb None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:34.416111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736087db-01ed-4f1b-9bd8-a80283bd67cb None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:34.416393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736087db-01ed-4f1b-9bd8-a80283bd67cb None admin] PATH_INFO: `/v3/users/6827d81000d9402b8bf49e5bf4aed9cd` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:34.417752 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736087db-01ed-4f1b-9bd8-a80283bd67cb None admin] RBAC: Authorizing `identity:get_user(user_id=6827d81000d9402b8bf49e5bf4aed9cd)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:34.420302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736087db-01ed-4f1b-9bd8-a80283bd67cb None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:34.422643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 69/131] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:34 2022] GET /identity/v3/users/6827d81000d9402b8bf49e5bf4aed9cd => generated 275 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:34.424199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736087db-01ed-4f1b-9bd8-a80283bd67cb None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:34.432643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373a1cd-b8ac-49ee-acba-a8df39af57ef None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:34.491890 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b820e5-370e-4a56-84f6-57265e4d3b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:34.492556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b820e5-370e-4a56-84f6-57265e4d3b44 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:34.492842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b820e5-370e-4a56-84f6-57265e4d3b44 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:34.493121 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b820e5-370e-4a56-84f6-57265e4d3b44 None admin] PATH_INFO: `/v3/users/6827d81000d9402b8bf49e5bf4aed9cd` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:34.498776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b820e5-370e-4a56-84f6-57265e4d3b44 None admin] RBAC: Authorizing `identity:get_user(user_id=6827d81000d9402b8bf49e5bf4aed9cd)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:34.500819 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b820e5-370e-4a56-84f6-57265e4d3b44 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:34.505711 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 63/132] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:34 2022] GET /identity/v3/users/6827d81000d9402b8bf49e5bf4aed9cd => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:34.506570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b820e5-370e-4a56-84f6-57265e4d3b44 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:34.516265 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:34.607529 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736087db-01ed-4f1b-9bd8-a80283bd67cb None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:34.610053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:34.666644 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fbc4a-7e93-402a-adc9-66951634ce9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:34.667761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fbc4a-7e93-402a-adc9-66951634ce9c None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:34.668259 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fbc4a-7e93-402a-adc9-66951634ce9c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:34.668754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fbc4a-7e93-402a-adc9-66951634ce9c None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4/users/6827d81000d9402b8bf49e5bf4aed9cd/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:34.673482 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fbc4a-7e93-402a-adc9-66951634ce9c None admin] RBAC: Authorizing `identity:create_grant(project_id=bc07ff985d9a458a94b1aefcf6147ae4, user_id=6827d81000d9402b8bf49e5bf4aed9cd, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:34.675339 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fbc4a-7e93-402a-adc9-66951634ce9c None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:34.690993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b820e5-370e-4a56-84f6-57265e4d3b44 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:34.728381 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19ad32-d16f-43b9-a8eb-86408c760465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:34.729373 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19ad32-d16f-43b9-a8eb-86408c760465 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:34.729913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19ad32-d16f-43b9-a8eb-86408c760465 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:34.730505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19ad32-d16f-43b9-a8eb-86408c760465 None admin] PATH_INFO: `/v3/projects/b7a794852e15470b8aa1f62923197266` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:34.731947 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19ad32-d16f-43b9-a8eb-86408c760465 None admin] RBAC: Authorizing `identity:get_project(project_id=b7a794852e15470b8aa1f62923197266)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:34.734242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19ad32-d16f-43b9-a8eb-86408c760465 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:34.736213 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 64/133] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:34 2022] GET /identity/v3/projects/b7a794852e15470b8aa1f62923197266 => generated 312 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:34.737228 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19ad32-d16f-43b9-a8eb-86408c760465 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:34.772065 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 70/134] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:37:34 2022] PUT /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4/users/6827d81000d9402b8bf49e5bf4aed9cd/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:37:34.773226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fbc4a-7e93-402a-adc9-66951634ce9c None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:34.937918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19ad32-d16f-43b9-a8eb-86408c760465 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:34.951823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fbc4a-7e93-402a-adc9-66951634ce9c None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:34.960769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:34.962469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:34.977865 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ddbbb-2121-4c6b-9837-f38b91f6d6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:34.978768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ddbbb-2121-4c6b-9837-f38b91f6d6cc None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:34.979247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ddbbb-2121-4c6b-9837-f38b91f6d6cc None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:34.979714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ddbbb-2121-4c6b-9837-f38b91f6d6cc None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:34.981058 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ddbbb-2121-4c6b-9837-f38b91f6d6cc None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:34.982835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ddbbb-2121-4c6b-9837-f38b91f6d6cc None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:34.985461 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 65/135] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:34 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:34.986835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b7eec-71b0-41a3-9525-0bb8c8e5ecf2 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:34.987311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b7eec-71b0-41a3-9525-0bb8c8e5ecf2 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:34.987722 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b7eec-71b0-41a3-9525-0bb8c8e5ecf2 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:34.989222 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 66/136] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:34.991395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b7eec-71b0-41a3-9525-0bb8c8e5ecf2 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:34.996460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81441818-1d70-4d7a-a108-02b671a69da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:34.999278 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81441818-1d70-4d7a-a108-02b671a69da9 None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:34.999658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81441818-1d70-4d7a-a108-02b671a69da9 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:34.999998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81441818-1d70-4d7a-a108-02b671a69da9 None admin] PATH_INFO: `/v3/projects/b7a794852e15470b8aa1f62923197266/users/6827d81000d9402b8bf49e5bf4aed9cd/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:35.002711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81441818-1d70-4d7a-a108-02b671a69da9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7a794852e15470b8aa1f62923197266, user_id=6827d81000d9402b8bf49e5bf4aed9cd, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:35.004314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81441818-1d70-4d7a-a108-02b671a69da9 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:35.069322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 71/137] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:37:34 2022] PUT /identity/v3/projects/b7a794852e15470b8aa1f62923197266/users/6827d81000d9402b8bf49e5bf4aed9cd/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:37:35.070759 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931331a-af0c-4210-80dc-a15c29caee88 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:35.071206 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931331a-af0c-4210-80dc-a15c29caee88 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:35.071496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931331a-af0c-4210-80dc-a15c29caee88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:35.120936 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-5931331a-af0c-4210-80dc-a15c29caee88 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:35.121556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-5931331a-af0c-4210-80dc-a15c29caee88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:35.131881 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b7eec-71b0-41a3-9525-0bb8c8e5ecf2 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:35.173103 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44618e-1bd3-403b-a007-7d5a82ae9b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:35.174093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44618e-1bd3-403b-a007-7d5a82ae9b8a None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:35.174556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44618e-1bd3-403b-a007-7d5a82ae9b8a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:35.174975 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44618e-1bd3-403b-a007-7d5a82ae9b8a None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4/users/6827d81000d9402b8bf49e5bf4aed9cd/roles/6fd5089acea448219d4c0a9a9424a3af` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:35.177787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44618e-1bd3-403b-a007-7d5a82ae9b8a None admin] RBAC: Authorizing `identity:create_grant(project_id=bc07ff985d9a458a94b1aefcf6147ae4, user_id=6827d81000d9402b8bf49e5bf4aed9cd, role_id=6fd5089acea448219d4c0a9a9424a3af)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:35.179593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44618e-1bd3-403b-a007-7d5a82ae9b8a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:35.230476 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 72/138] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:35.257072 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 67/139] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:37:34 2022] PUT /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4/users/6827d81000d9402b8bf49e5bf4aed9cd/roles/6fd5089acea448219d4c0a9a9424a3af => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:37:35.494912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a0013-0ff4-40d5-870c-60ca05e1b5da None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:35.495288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a0013-0ff4-40d5-870c-60ca05e1b5da None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:35.495464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a0013-0ff4-40d5-870c-60ca05e1b5da None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:35.497167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 73/140] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:35.502605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18938e-5d04-4aa6-a32c-dd6bfa55452a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:35.502897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18938e-5d04-4aa6-a32c-dd6bfa55452a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:35.503254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18938e-5d04-4aa6-a32c-dd6bfa55452a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:35.532177 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6e18938e-5d04-4aa6-a32c-dd6bfa55452a None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:35.533398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6e18938e-5d04-4aa6-a32c-dd6bfa55452a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:35.557516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a0013-0ff4-40d5-870c-60ca05e1b5da None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:35.584192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:35.621793 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 68/141] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:35.672073 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:35.676706 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a0013-0ff4-40d5-870c-60ca05e1b5da None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:35.712992 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba201e19-a6ca-46c0-9b72-a615abd8ee4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:35.713682 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba201e19-a6ca-46c0-9b72-a615abd8ee4e None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:35.713972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba201e19-a6ca-46c0-9b72-a615abd8ee4e None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:35.714243 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba201e19-a6ca-46c0-9b72-a615abd8ee4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:35.715398 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba201e19-a6ca-46c0-9b72-a615abd8ee4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:35.716909 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba201e19-a6ca-46c0-9b72-a615abd8ee4e None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:35.719669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 74/142] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:35.725333 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18938e-5d04-4aa6-a32c-dd6bfa55452a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:35.851286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18938e-5d04-4aa6-a32c-dd6bfa55452a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:35.895792 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c03d1-d02e-4136-bbab-43ab1162dfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:35.896415 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c03d1-d02e-4136-bbab-43ab1162dfd8 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:35.896698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c03d1-d02e-4136-bbab-43ab1162dfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:35.896987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c03d1-d02e-4136-bbab-43ab1162dfd8 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:35.897666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c03d1-d02e-4136-bbab-43ab1162dfd8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:35.899168 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c03d1-d02e-4136-bbab-43ab1162dfd8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:35.984405 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 69/143] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:37:35 2022] POST /identity/v3/users => generated 283 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:37:36.224074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba201e19-a6ca-46c0-9b72-a615abd8ee4e None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:36.344990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba201e19-a6ca-46c0-9b72-a615abd8ee4e None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:36.382972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d65427-e18c-4425-9c2c-3c957b1f7128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:36.383643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d65427-e18c-4425-9c2c-3c957b1f7128 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:36.383929 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d65427-e18c-4425-9c2c-3c957b1f7128 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:36.384218 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d65427-e18c-4425-9c2c-3c957b1f7128 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:36.395151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d65427-e18c-4425-9c2c-3c957b1f7128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:36.400138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-40d65427-e18c-4425-9c2c-3c957b1f7128 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:37:36.400138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:36.400138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:36.400138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:36.400138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:36.400138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:36.400138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:36.400138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:36.400138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:36.400138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:36.400138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:36.400138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:36.400138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:36.400138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:36.400138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:36.400138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:36.400976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:36.400976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:36.400976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:36.400976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:36.400976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:36.400976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:36.400976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:36.400976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:36.400976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:36.400976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:36.400976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:36.400976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:36.400976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:36.400976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:36.400976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:36.400976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:36.401717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:36.401717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:36.401717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:36.401717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:36.401717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:36.401717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:36.401717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:36.401717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:36.401717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:36.401717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:36.401717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:37:36.401717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:37:36.401717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 75/144] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:36.407896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c03d1-d02e-4136-bbab-43ab1162dfd8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:36.444659 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:36.530165 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c03d1-d02e-4136-bbab-43ab1162dfd8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:36.569872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc838cdc-3c26-4d9b-8552-6bb95afa85e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:36.570497 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc838cdc-3c26-4d9b-8552-6bb95afa85e2 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:36.570776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc838cdc-3c26-4d9b-8552-6bb95afa85e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:36.571053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc838cdc-3c26-4d9b-8552-6bb95afa85e2 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:36.571772 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc838cdc-3c26-4d9b-8552-6bb95afa85e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:36.572085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc838cdc-3c26-4d9b-8552-6bb95afa85e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:36.573478 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc838cdc-3c26-4d9b-8552-6bb95afa85e2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:36.586996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 70/145] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:36.592499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d65427-e18c-4425-9c2c-3c957b1f7128 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:36.624478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:36.705804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d65427-e18c-4425-9c2c-3c957b1f7128 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:36.747388 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec35fc7-e5ff-4f7e-b023-9add31bb8311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:36.748068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec35fc7-e5ff-4f7e-b023-9add31bb8311 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:36.748349 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec35fc7-e5ff-4f7e-b023-9add31bb8311 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:36.749243 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec35fc7-e5ff-4f7e-b023-9add31bb8311 None admin] PATH_INFO: `/v3/users/ce5c8b94724b444f90b8cf10caac2c1a` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:36.750490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec35fc7-e5ff-4f7e-b023-9add31bb8311 None admin] RBAC: Authorizing `identity:get_user(user_id=ce5c8b94724b444f90b8cf10caac2c1a)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:36.752052 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec35fc7-e5ff-4f7e-b023-9add31bb8311 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:36.753735 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 76/146] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:36 2022] GET /identity/v3/users/ce5c8b94724b444f90b8cf10caac2c1a => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:36.759339 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc838cdc-3c26-4d9b-8552-6bb95afa85e2 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:36.788509 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fd319-8f11-4b3c-970b-cc4ad5a15637 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:36.788776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fd319-8f11-4b3c-970b-cc4ad5a15637 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:36.788978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fd319-8f11-4b3c-970b-cc4ad5a15637 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:36.790646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 77/147] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:36.797169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c42cde-426a-4c37-a46e-9ba8726b1f87 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:36.797448 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c42cde-426a-4c37-a46e-9ba8726b1f87 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:36.797720 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c42cde-426a-4c37-a46e-9ba8726b1f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:36.825762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-b3c42cde-426a-4c37-a46e-9ba8726b1f87 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:36.827013 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b3c42cde-426a-4c37-a46e-9ba8726b1f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:36.922807 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:36.926231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 78/148] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:37:36.933216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc838cdc-3c26-4d9b-8552-6bb95afa85e2 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:36.982554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f3b68-0598-453c-86a2-05499443365f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:36.983794 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f3b68-0598-453c-86a2-05499443365f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:36.984079 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f3b68-0598-453c-86a2-05499443365f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:36.984521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f3b68-0598-453c-86a2-05499443365f None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:36.989067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f3b68-0598-453c-86a2-05499443365f None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:36.990549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f3b68-0598-453c-86a2-05499443365f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:36.992563 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 71/149] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:36 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:36.998008 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c42cde-426a-4c37-a46e-9ba8726b1f87 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:37.037742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:37.128980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c42cde-426a-4c37-a46e-9ba8726b1f87 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:37.166925 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0471215-5fe6-48d6-8b79-3ea23e1e2a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:37.167664 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0471215-5fe6-48d6-8b79-3ea23e1e2a94 None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:37.167942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0471215-5fe6-48d6-8b79-3ea23e1e2a94 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:37.168215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0471215-5fe6-48d6-8b79-3ea23e1e2a94 None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4/users/ce5c8b94724b444f90b8cf10caac2c1a/roles/486acd6fe1174148b9461c56c9d371aa` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:37.170181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0471215-5fe6-48d6-8b79-3ea23e1e2a94 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc07ff985d9a458a94b1aefcf6147ae4, user_id=ce5c8b94724b444f90b8cf10caac2c1a, role_id=486acd6fe1174148b9461c56c9d371aa)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:37.172228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0471215-5fe6-48d6-8b79-3ea23e1e2a94 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:37.247032 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 79/150] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:37:36 2022] PUT /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4/users/ce5c8b94724b444f90b8cf10caac2c1a/roles/486acd6fe1174148b9461c56c9d371aa => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:37:37.279822 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f3b68-0598-453c-86a2-05499443365f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:37.424108 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f3b68-0598-453c-86a2-05499443365f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:37.465163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498427a-cd5d-4d3c-a694-92d149d64ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:37.465774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bcf53-9f28-4780-a716-02ad850bbcc7 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:37.466191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498427a-cd5d-4d3c-a694-92d149d64ed1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:37.466262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bcf53-9f28-4780-a716-02ad850bbcc7 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:37.466325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498427a-cd5d-4d3c-a694-92d149d64ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:37.466386 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bcf53-9f28-4780-a716-02ad850bbcc7 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:37.466459 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498427a-cd5d-4d3c-a694-92d149d64ed1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:37.468504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 80/151] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:37.473970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c657e-229a-4fd8-86b6-6181eb6186bf None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:37.474530 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c657e-229a-4fd8-86b6-6181eb6186bf None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:37.474818 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c657e-229a-4fd8-86b6-6181eb6186bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:37.486621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498427a-cd5d-4d3c-a694-92d149d64ed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:37.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-6498427a-cd5d-4d3c-a694-92d149d64ed1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:37.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:37.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:37.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:37.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:37.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:37.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:37.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:37.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:37.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:37.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:37.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:37.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:37.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:37.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:37.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:37.492464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:37.492464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:37.492464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:37.492464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:37.492464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:37.492464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:37.492464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:37.492464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:37.492464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:37.492464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:37.492464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:37.492464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:37.492464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:37.492464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:37.492464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:37.492464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:37.493211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:37.493211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:37.493211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:37.493211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:37.493211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:37.493211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:37.493211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:37.493211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:37.493211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:37.493211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:37.493211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:37.493211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:37.493211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 72/152] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:37:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:37.499428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498427a-cd5d-4d3c-a694-92d149d64ed1 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:37.512584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-0e4c657e-229a-4fd8-86b6-6181eb6186bf None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:37.514523 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0e4c657e-229a-4fd8-86b6-6181eb6186bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:37.545067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:37.636853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 81/153] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:37:37.660721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498427a-cd5d-4d3c-a694-92d149d64ed1 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:37.706859 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078f490-fd6e-4c4c-9f17-e18ec34f486e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:37.707724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078f490-fd6e-4c4c-9f17-e18ec34f486e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:37.708171 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078f490-fd6e-4c4c-9f17-e18ec34f486e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:37.708587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078f490-fd6e-4c4c-9f17-e18ec34f486e None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:37.709412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078f490-fd6e-4c4c-9f17-e18ec34f486e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:37.709857 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078f490-fd6e-4c4c-9f17-e18ec34f486e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:37.711441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078f490-fd6e-4c4c-9f17-e18ec34f486e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:37.720856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 73/154] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:37.740620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c657e-229a-4fd8-86b6-6181eb6186bf None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:37.760399 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:37.870102 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c657e-229a-4fd8-86b6-6181eb6186bf None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:37.914693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcf9fb-9543-40d7-b1fa-6f5de49ae581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:37.915403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcf9fb-9543-40d7-b1fa-6f5de49ae581 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:37.915686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcf9fb-9543-40d7-b1fa-6f5de49ae581 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:37.915963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcf9fb-9543-40d7-b1fa-6f5de49ae581 None admin] PATH_INFO: `/v3/users/ef040204e84943a7ad9a173f7dd9bb2b` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:37.917128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcf9fb-9543-40d7-b1fa-6f5de49ae581 None admin] RBAC: Authorizing `identity:get_user(user_id=ef040204e84943a7ad9a173f7dd9bb2b)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:37.918641 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcf9fb-9543-40d7-b1fa-6f5de49ae581 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:37.920681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 82/155] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:37 2022] GET /identity/v3/users/ef040204e84943a7ad9a173f7dd9bb2b => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:37.926177 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078f490-fd6e-4c4c-9f17-e18ec34f486e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:38.039827 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:38.057152 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078f490-fd6e-4c4c-9f17-e18ec34f486e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:38.095810 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c02700-47aa-4696-a1d9-fd3cdd69ee37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:38.096439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c02700-47aa-4696-a1d9-fd3cdd69ee37 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:38.096725 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c02700-47aa-4696-a1d9-fd3cdd69ee37 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:38.097011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c02700-47aa-4696-a1d9-fd3cdd69ee37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:38.133579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcf9fb-9543-40d7-b1fa-6f5de49ae581 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:38.191241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c02700-47aa-4696-a1d9-fd3cdd69ee37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:38.191241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c02700-47aa-4696-a1d9-fd3cdd69ee37 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:38.275616 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:38.303252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcf9fb-9543-40d7-b1fa-6f5de49ae581 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:38.314761 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:38.339652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 74/156] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:38.341175 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e0252-76de-42d9-b07f-aa36d1d2922b None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:38.341467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e0252-76de-42d9-b07f-aa36d1d2922b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:38.341746 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e0252-76de-42d9-b07f-aa36d1d2922b None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:38.343184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 75/157] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:38.347971 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e0252-76de-42d9-b07f-aa36d1d2922b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:38.352748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80358c89-0e42-4372-8620-21158f4e19f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:38.353624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80358c89-0e42-4372-8620-21158f4e19f4 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:38.354133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80358c89-0e42-4372-8620-21158f4e19f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:38.354579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80358c89-0e42-4372-8620-21158f4e19f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:38.372620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80358c89-0e42-4372-8620-21158f4e19f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:38.387353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-80358c89-0e42-4372-8620-21158f4e19f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:38.387353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:38.387353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:38.387353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:38.387353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:38.387353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:38.387353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:38.387353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:38.387353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:38.387353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:38.387353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:38.387353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:38.387353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:38.387353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:38.387353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:38.387353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:38.388261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:38.388261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:38.388261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:38.388261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:38.388261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:38.388261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:38.388261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:38.388261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:38.388261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:38.388261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:38.388261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:38.388261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:38.388261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:38.388261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:38.388261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:38.388261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:38.389049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:38.389049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:38.389049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:38.389049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:38.389049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:38.389049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:38.389049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:38.389049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:38.389049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:38.389049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:38.389049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:38.389049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:37:38.391744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 83/158] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:38.393257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a190dc-0e00-4e02-9864-9c12b88dc029 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:38.393718 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a190dc-0e00-4e02-9864-9c12b88dc029 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:38.394159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a190dc-0e00-4e02-9864-9c12b88dc029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:38.424767 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-35a190dc-0e00-4e02-9864-9c12b88dc029 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:38.426406 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-35a190dc-0e00-4e02-9864-9c12b88dc029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:38.495799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e0252-76de-42d9-b07f-aa36d1d2922b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:38.532382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 84/159] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:38.533230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a190dc-0e00-4e02-9864-9c12b88dc029 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:38.559917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81489b47-c3dd-412e-949e-5c1dcf82e53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:38.560893 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81489b47-c3dd-412e-949e-5c1dcf82e53c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:38.561441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81489b47-c3dd-412e-949e-5c1dcf82e53c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:38.561911 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81489b47-c3dd-412e-949e-5c1dcf82e53c None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:38.563337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81489b47-c3dd-412e-949e-5c1dcf82e53c None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:38.565078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81489b47-c3dd-412e-949e-5c1dcf82e53c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:38.566950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 76/160] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:38 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:38.572889 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81489b47-c3dd-412e-949e-5c1dcf82e53c None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:38.653705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:38.663135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:38.683549 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a190dc-0e00-4e02-9864-9c12b88dc029 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:38.722841 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81489b47-c3dd-412e-949e-5c1dcf82e53c None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:38.738171 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0398ccb-dd72-4eb5-9212-48b9a827fcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:38.739202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0398ccb-dd72-4eb5-9212-48b9a827fcbd None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:38.739661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0398ccb-dd72-4eb5-9212-48b9a827fcbd None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:38.740132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0398ccb-dd72-4eb5-9212-48b9a827fcbd None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:38.741093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0398ccb-dd72-4eb5-9212-48b9a827fcbd None admin] RBAC: Adding query filter params (name=member) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:38.741573 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0398ccb-dd72-4eb5-9212-48b9a827fcbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:38.743258 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0398ccb-dd72-4eb5-9212-48b9a827fcbd None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:38.756990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 85/161] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:38.762988 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0398ccb-dd72-4eb5-9212-48b9a827fcbd None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:38.781286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ece1e4-eb06-437c-bb89-14677265ef42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:38.782121 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ece1e4-eb06-437c-bb89-14677265ef42 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:38.782529 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ece1e4-eb06-437c-bb89-14677265ef42 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:38.782929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ece1e4-eb06-437c-bb89-14677265ef42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:38.783976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ece1e4-eb06-437c-bb89-14677265ef42 None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4, user.id=ef040204e84943a7ad9a173f7dd9bb2b) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:38.784468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ece1e4-eb06-437c-bb89-14677265ef42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:38.786099 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ece1e4-eb06-437c-bb89-14677265ef42 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:38.800682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 77/162] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:38 2022] GET /identity/v3/role_assignments?user.id=ef040204e84943a7ad9a173f7dd9bb2b&scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:38.911946 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0398ccb-dd72-4eb5-9212-48b9a827fcbd None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:38.935152 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c853c-bfca-47e7-9f93-9f802579a2d3 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:38.935774 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c853c-bfca-47e7-9f93-9f802579a2d3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:38.936294 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c853c-bfca-47e7-9f93-9f802579a2d3 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:38.937894 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 78/163] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:38.952365 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:38.953829 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac9fe0-31d0-4674-8261-5c42e1f8ca76 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:38.954356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac9fe0-31d0-4674-8261-5c42e1f8ca76 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:38.954830 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac9fe0-31d0-4674-8261-5c42e1f8ca76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:38.963152 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e953c-2f04-4b74-b02f-2e6c57e9a23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:38.963152 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e953c-2f04-4b74-b02f-2e6c57e9a23b None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:38.963152 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e953c-2f04-4b74-b02f-2e6c57e9a23b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:38.963152 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e953c-2f04-4b74-b02f-2e6c57e9a23b None admin] PATH_INFO: `/v3/users/6827d81000d9402b8bf49e5bf4aed9cd` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:38.965228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e953c-2f04-4b74-b02f-2e6c57e9a23b None admin] RBAC: Authorizing `identity:get_user(user_id=6827d81000d9402b8bf49e5bf4aed9cd)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:38.966776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e953c-2f04-4b74-b02f-2e6c57e9a23b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:38.969379 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 86/164] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:38 2022] GET /identity/v3/users/6827d81000d9402b8bf49e5bf4aed9cd => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:38.979146 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e953c-2f04-4b74-b02f-2e6c57e9a23b None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:39.000137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-77ac9fe0-31d0-4674-8261-5c42e1f8ca76 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:39.001693 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-77ac9fe0-31d0-4674-8261-5c42e1f8ca76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:39.123912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 79/165] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:39.125114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac9fe0-31d0-4674-8261-5c42e1f8ca76 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:39.210642 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:39.225981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e953c-2f04-4b74-b02f-2e6c57e9a23b None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:39.277047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac9fe0-31d0-4674-8261-5c42e1f8ca76 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:39.331813 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c99a58-30b4-4f5c-a73e-a15231353b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:39.332014 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae9670-0d6b-4e6c-99d2-7aa21c33b4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:39.332439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c99a58-30b4-4f5c-a73e-a15231353b26 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:39.332544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae9670-0d6b-4e6c-99d2-7aa21c33b4ad None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:39.332718 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c99a58-30b4-4f5c-a73e-a15231353b26 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:39.332814 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae9670-0d6b-4e6c-99d2-7aa21c33b4ad None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:39.332987 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c99a58-30b4-4f5c-a73e-a15231353b26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:39.333102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae9670-0d6b-4e6c-99d2-7aa21c33b4ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:39.347167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae9670-0d6b-4e6c-99d2-7aa21c33b4ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:39.352864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-79ae9670-0d6b-4e6c-99d2-7aa21c33b4ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:39.352864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:39.352864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:39.352864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:39.352864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:39.352864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:39.352864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:39.352864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:39.352864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:39.352864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:39.352864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:39.352864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:39.352864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:39.352864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:39.352864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:39.352864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:39.353729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:39.353729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:39.353729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:39.353729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:39.353729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:39.353729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:39.353729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:39.353729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:39.353729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:39.353729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:39.353729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:39.353729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:39.353729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:39.353729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:39.353729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:39.353729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:39.354688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:39.354688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:39.354688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:39.354688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:39.354688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:39.354688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:39.354688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:39.354688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:39.354688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:39.354688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:39.354688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:39.354688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:39.354688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 80/166] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:39.360174 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae9670-0d6b-4e6c-99d2-7aa21c33b4ad None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:39.526087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:39.527976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae9670-0d6b-4e6c-99d2-7aa21c33b4ad None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:39.532288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c99a58-30b4-4f5c-a73e-a15231353b26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:39.533753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c99a58-30b4-4f5c-a73e-a15231353b26 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:39.555450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:39.604540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c6994-07f4-4e7b-90c9-08a9662006b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:39.605152 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c6994-07f4-4e7b-90c9-08a9662006b5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:39.605430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c6994-07f4-4e7b-90c9-08a9662006b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:39.605698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c6994-07f4-4e7b-90c9-08a9662006b5 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:39.606372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c6994-07f4-4e7b-90c9-08a9662006b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:39.606675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c6994-07f4-4e7b-90c9-08a9662006b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:39.611067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c6994-07f4-4e7b-90c9-08a9662006b5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:39.631371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 81/167] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:39.637036 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c6994-07f4-4e7b-90c9-08a9662006b5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:39.738449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 87/168] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 764 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:39.744164 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c99a58-30b4-4f5c-a73e-a15231353b26 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:39.821472 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c6994-07f4-4e7b-90c9-08a9662006b5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:39.892932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:39.896697 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c99a58-30b4-4f5c-a73e-a15231353b26 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:39.912381 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e5533-33b8-4398-83b4-5009578ffede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:39.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5533-33b8-4398-83b4-5009578ffede None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:39.913389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5533-33b8-4398-83b4-5009578ffede None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:39.913664 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5533-33b8-4398-83b4-5009578ffede None admin] PATH_INFO: `/v3/users/6827d81000d9402b8bf49e5bf4aed9cd` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:39.914836 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e5533-33b8-4398-83b4-5009578ffede None admin] RBAC: Authorizing `identity:get_user(user_id=6827d81000d9402b8bf49e5bf4aed9cd)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:39.916351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e5533-33b8-4398-83b4-5009578ffede None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:39.918295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 82/169] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:39 2022] GET /identity/v3/users/6827d81000d9402b8bf49e5bf4aed9cd => generated 275 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:39.919250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e5533-33b8-4398-83b4-5009578ffede None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:39.941022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7fb68-e4ea-40d1-a006-71532dbf8f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:39.942046 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7fb68-e4ea-40d1-a006-71532dbf8f3e None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:39.942496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7fb68-e4ea-40d1-a006-71532dbf8f3e None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:39.942937 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7fb68-e4ea-40d1-a006-71532dbf8f3e None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:39.944315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7fb68-e4ea-40d1-a006-71532dbf8f3e None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:39.946090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7fb68-e4ea-40d1-a006-71532dbf8f3e None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:39.951004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 88/170] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:39 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:39.952810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7fb68-e4ea-40d1-a006-71532dbf8f3e None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:39.997478 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:40.117242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e5533-33b8-4398-83b4-5009578ffede None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:40.138474 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7fb68-e4ea-40d1-a006-71532dbf8f3e None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:40.183693 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7ea0d-6277-4392-a6c6-164f9184e7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:40.185066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7ea0d-6277-4392-a6c6-164f9184e7f7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:40.185646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7ea0d-6277-4392-a6c6-164f9184e7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:40.186945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7ea0d-6277-4392-a6c6-164f9184e7f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:40.206016 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f299bf-5488-4150-96cb-242c7d6d9c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:40.206954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f299bf-5488-4150-96cb-242c7d6d9c85 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:40.207665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f299bf-5488-4150-96cb-242c7d6d9c85 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:40.208321 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7ea0d-6277-4392-a6c6-164f9184e7f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:40.208764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f299bf-5488-4150-96cb-242c7d6d9c85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:40.214344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-cdb7ea0d-6277-4392-a6c6-164f9184e7f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:37:40.214344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:40.214344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:40.214344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:40.214344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:40.214344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:40.214344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:40.214344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:40.214344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:40.214344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:40.214344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:40.214344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:40.214344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:40.214344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:40.214344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:40.214344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:40.215240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:40.215240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:40.215240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:40.215240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:40.215240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:40.215240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:40.215240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:40.215240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:40.215240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:40.215240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:40.215240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:40.215240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:40.215240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:40.215240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:40.215240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:40.215240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:40.216019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:40.216019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:40.216019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:40.216019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:40.216019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:40.216019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:40.216019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:40.216019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:40.216019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:40.216019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:40.216019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:37:40.216019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:40.219012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 83/171] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:37:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:40.221978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7ea0d-6277-4392-a6c6-164f9184e7f7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:40.303562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:40.381355 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f299bf-5488-4150-96cb-242c7d6d9c85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:40.382755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f299bf-5488-4150-96cb-242c7d6d9c85 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:40.413496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:40.418334 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7ea0d-6277-4392-a6c6-164f9184e7f7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:40.458583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdf0b9-50da-4bed-8e78-3f75d5740099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:40.459693 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdf0b9-50da-4bed-8e78-3f75d5740099 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:40.460130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdf0b9-50da-4bed-8e78-3f75d5740099 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:40.460541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdf0b9-50da-4bed-8e78-3f75d5740099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:40.461466 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fdf0b9-50da-4bed-8e78-3f75d5740099 None admin] RBAC: Adding query filter params (role.id=7c4463788f534454b6d8319dbc4d2afc, scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4, user.id=6827d81000d9402b8bf49e5bf4aed9cd) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:40.461957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fdf0b9-50da-4bed-8e78-3f75d5740099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:40.463571 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fdf0b9-50da-4bed-8e78-3f75d5740099 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:40.470810 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 84/172] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:40 2022] GET /identity/v3/role_assignments?user.id=6827d81000d9402b8bf49e5bf4aed9cd&scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4&role.id=7c4463788f534454b6d8319dbc4d2afc => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:40.471985 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdf0b9-50da-4bed-8e78-3f75d5740099 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:40.578096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 89/173] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:40.583747 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f299bf-5488-4150-96cb-242c7d6d9c85 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:40.588244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdf0b9-50da-4bed-8e78-3f75d5740099 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:40.628139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdacd3-69db-4c4b-9795-900dc86971d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:40.629102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdacd3-69db-4c4b-9795-900dc86971d9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:40.629552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdacd3-69db-4c4b-9795-900dc86971d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:40.629987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdacd3-69db-4c4b-9795-900dc86971d9 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:40.631029 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fdacd3-69db-4c4b-9795-900dc86971d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:40.631582 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fdacd3-69db-4c4b-9795-900dc86971d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:40.633152 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fdacd3-69db-4c4b-9795-900dc86971d9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:40.642401 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 85/174] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:37:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:40.652547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdacd3-69db-4c4b-9795-900dc86971d9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:40.694750 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:40.734084 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:40.793243 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f299bf-5488-4150-96cb-242c7d6d9c85 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:40.798520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdacd3-69db-4c4b-9795-900dc86971d9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:40.835858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2ade9-d751-4e51-8165-d1015fee8bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:40.836789 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2ade9-d751-4e51-8165-d1015fee8bed None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:40.837263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2ade9-d751-4e51-8165-d1015fee8bed None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:40.837802 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2ade9-d751-4e51-8165-d1015fee8bed None admin] PATH_INFO: `/v3/users/6827d81000d9402b8bf49e5bf4aed9cd` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:40.839250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2ade9-d751-4e51-8165-d1015fee8bed None admin] RBAC: Authorizing `identity:get_user(user_id=6827d81000d9402b8bf49e5bf4aed9cd)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:40.840876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2ade9-d751-4e51-8165-d1015fee8bed None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:40.843286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 86/175] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:40 2022] GET /identity/v3/users/6827d81000d9402b8bf49e5bf4aed9cd => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:40.844713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540fdeb-5612-431c-966b-aac576525001 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:40.845261 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540fdeb-5612-431c-966b-aac576525001 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:40.845829 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540fdeb-5612-431c-966b-aac576525001 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:40.847223 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 87/176] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:40.855274 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230cd4c-ca42-462d-84c1-5c7171e8a0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:40.855454 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9588ef-916a-47c8-8d2b-392f48c319d5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:40.855720 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9588ef-916a-47c8-8d2b-392f48c319d5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:40.855917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230cd4c-ca42-462d-84c1-5c7171e8a0b5 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:40.856017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9588ef-916a-47c8-8d2b-392f48c319d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:40.856193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230cd4c-ca42-462d-84c1-5c7171e8a0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:40.856459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230cd4c-ca42-462d-84c1-5c7171e8a0b5 None admin] PATH_INFO: `/v3/projects/b7a794852e15470b8aa1f62923197266` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:40.858420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230cd4c-ca42-462d-84c1-5c7171e8a0b5 None admin] RBAC: Authorizing `identity:get_project(project_id=b7a794852e15470b8aa1f62923197266)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:40.859906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230cd4c-ca42-462d-84c1-5c7171e8a0b5 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:40.861543 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 90/177] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:40 2022] GET /identity/v3/projects/b7a794852e15470b8aa1f62923197266 => generated 312 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:40.862292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230cd4c-ca42-462d-84c1-5c7171e8a0b5 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:40.880093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-fd9588ef-916a-47c8-8d2b-392f48c319d5 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:40.881284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fd9588ef-916a-47c8-8d2b-392f48c319d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:40.967844 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 88/178] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:40.969238 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4677373-863c-4b59-80a1-9eeb51340bdf None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:40.969528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4677373-863c-4b59-80a1-9eeb51340bdf None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:40.969910 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4677373-863c-4b59-80a1-9eeb51340bdf None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:40.971233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 89/179] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:40.972098 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4677373-863c-4b59-80a1-9eeb51340bdf None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:40.979645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230cd4c-ca42-462d-84c1-5c7171e8a0b5 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:41.007217 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:41.020846 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e909dd1-33d5-4cae-8d96-987fc0ebef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:41.021458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e909dd1-33d5-4cae-8d96-987fc0ebef57 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:41.021733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e909dd1-33d5-4cae-8d96-987fc0ebef57 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:41.022004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e909dd1-33d5-4cae-8d96-987fc0ebef57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:41.025772 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:41.097506 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e909dd1-33d5-4cae-8d96-987fc0ebef57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:41.099061 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e909dd1-33d5-4cae-8d96-987fc0ebef57 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:41.109095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4677373-863c-4b59-80a1-9eeb51340bdf None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:41.147122 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9346419-4c80-4af6-ac49-ede670b0320b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:41.147854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9346419-4c80-4af6-ac49-ede670b0320b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:41.148137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9346419-4c80-4af6-ac49-ede670b0320b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:41.148406 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9346419-4c80-4af6-ac49-ede670b0320b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:41.149257 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9346419-4c80-4af6-ac49-ede670b0320b None admin] RBAC: Adding query filter params (role.id=7c4463788f534454b6d8319dbc4d2afc, scope.project.id=b7a794852e15470b8aa1f62923197266, user.id=6827d81000d9402b8bf49e5bf4aed9cd) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:41.149560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9346419-4c80-4af6-ac49-ede670b0320b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:41.151040 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9346419-4c80-4af6-ac49-ede670b0320b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:41.157527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 90/180] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:40 2022] GET /identity/v3/role_assignments?user.id=6827d81000d9402b8bf49e5bf4aed9cd&scope.project.id=b7a794852e15470b8aa1f62923197266&role.id=7c4463788f534454b6d8319dbc4d2afc => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:41.164567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9325e10-c273-4aca-a7f5-c09293fb15ab None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:41.164840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9325e10-c273-4aca-a7f5-c09293fb15ab None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:41.165110 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9325e10-c273-4aca-a7f5-c09293fb15ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:41.191839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-d9325e10-c273-4aca-a7f5-c09293fb15ab None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:41.193040 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d9325e10-c273-4aca-a7f5-c09293fb15ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:41.207779 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 91/181] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:41.209146 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2520c3-2205-4a28-96c6-e7777ed93c87 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:41.209548 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2520c3-2205-4a28-96c6-e7777ed93c87 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:41.209826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2520c3-2205-4a28-96c6-e7777ed93c87 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:41.211452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 92/182] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:41.216572 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b08724-1fc6-447a-8575-a69940fc952c None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:41.216848 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b08724-1fc6-447a-8575-a69940fc952c None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:41.217156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b08724-1fc6-447a-8575-a69940fc952c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:41.242931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-86b08724-1fc6-447a-8575-a69940fc952c None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:41.242931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-86b08724-1fc6-447a-8575-a69940fc952c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:41.307473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:41.318591 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 93/183] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:37:41.325139 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b08724-1fc6-447a-8575-a69940fc952c None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:41.329110 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 91/184] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:41.379809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840daf6-4bac-434f-aec8-a7d331c9e2da None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:41.380150 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840daf6-4bac-434f-aec8-a7d331c9e2da None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:41.380355 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840daf6-4bac-434f-aec8-a7d331c9e2da None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:41.383202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 92/185] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:41.386640 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840daf6-4bac-434f-aec8-a7d331c9e2da None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:41.432310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:41.483797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b08724-1fc6-447a-8575-a69940fc952c None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:41.524495 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4b2db-3f33-4fd6-bd5c-4056ce722e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:41.525053 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4b2db-3f33-4fd6-bd5c-4056ce722e0a None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:41.525332 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4b2db-3f33-4fd6-bd5c-4056ce722e0a None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:41.525603 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4b2db-3f33-4fd6-bd5c-4056ce722e0a None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:41.526844 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4b2db-3f33-4fd6-bd5c-4056ce722e0a None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:41.528600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4b2db-3f33-4fd6-bd5c-4056ce722e0a None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:41.530378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 94/186] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:41 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:41.531765 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00040d31-784c-4ec5-805b-cfca599c1746 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:41.532100 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00040d31-784c-4ec5-805b-cfca599c1746 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:41.532384 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00040d31-784c-4ec5-805b-cfca599c1746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:41.543465 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840daf6-4bac-434f-aec8-a7d331c9e2da None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:41.553041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-00040d31-784c-4ec5-805b-cfca599c1746 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:41.554243 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-00040d31-784c-4ec5-805b-cfca599c1746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:41.586589 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:41.590081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbd4bb-97ad-4b13-8387-3d743bb7a41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:41.591482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbd4bb-97ad-4b13-8387-3d743bb7a41e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:41.591975 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbd4bb-97ad-4b13-8387-3d743bb7a41e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:41.592504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbd4bb-97ad-4b13-8387-3d743bb7a41e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:41.605087 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbd4bb-97ad-4b13-8387-3d743bb7a41e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:41.611741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-5acbd4bb-97ad-4b13-8387-3d743bb7a41e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:37:41.611741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:41.611741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:41.611741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:41.611741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:41.611741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:41.611741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:41.611741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:41.611741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:41.611741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:41.611741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:41.611741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:41.611741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:41.611741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:41.611741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:41.611741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:41.612632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:41.612632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:41.612632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:41.612632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:41.612632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:41.612632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:41.612632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:41.612632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:41.612632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:41.612632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:41.612632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:41.612632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:41.612632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:41.612632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:41.612632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:41.612632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:41.613420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:41.613420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:41.613420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:41.613420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:41.613420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:41.613420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:41.613420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:41.613420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:41.613420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:41.613420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:41.613420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:37:41.613420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:41.615552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 93/187] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:41.616454 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbd4bb-97ad-4b13-8387-3d743bb7a41e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:41.636118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 95/188] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:41.637118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00040d31-784c-4ec5-805b-cfca599c1746 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:41.709709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:41.730176 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbd4bb-97ad-4b13-8387-3d743bb7a41e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:41.751729 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00040d31-784c-4ec5-805b-cfca599c1746 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:41.765875 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd744f6-a947-48e8-b355-9e3e2fdbe212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:41.766576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd744f6-a947-48e8-b355-9e3e2fdbe212 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:41.766859 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd744f6-a947-48e8-b355-9e3e2fdbe212 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:41.767143 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd744f6-a947-48e8-b355-9e3e2fdbe212 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:41.767874 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd744f6-a947-48e8-b355-9e3e2fdbe212 None admin] RBAC: Adding query filter params (role.id=6fd5089acea448219d4c0a9a9424a3af, scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4, user.id=6827d81000d9402b8bf49e5bf4aed9cd) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:41.768177 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd744f6-a947-48e8-b355-9e3e2fdbe212 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:41.769750 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd744f6-a947-48e8-b355-9e3e2fdbe212 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:41.787427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 94/189] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:41 2022] GET /identity/v3/role_assignments?user.id=6827d81000d9402b8bf49e5bf4aed9cd&scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4&role.id=6fd5089acea448219d4c0a9a9424a3af => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:41.787427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd744f6-a947-48e8-b355-9e3e2fdbe212 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:41.803423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86784eda-dc8c-4bb5-9453-b71df0198d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:41.804039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86784eda-dc8c-4bb5-9453-b71df0198d5d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:41.804314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86784eda-dc8c-4bb5-9453-b71df0198d5d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:41.804581 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86784eda-dc8c-4bb5-9453-b71df0198d5d None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:41.805260 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86784eda-dc8c-4bb5-9453-b71df0198d5d None admin] RBAC: Adding query filter params (name=reader) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:41.805559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86784eda-dc8c-4bb5-9453-b71df0198d5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:41.806990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86784eda-dc8c-4bb5-9453-b71df0198d5d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:41.816142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 96/190] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:41.817005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86784eda-dc8c-4bb5-9453-b71df0198d5d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:41.846811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:41.897211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd744f6-a947-48e8-b355-9e3e2fdbe212 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:41.924596 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86784eda-dc8c-4bb5-9453-b71df0198d5d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:41.938103 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3f410-3f91-4fd9-a2b4-30e568bd5f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:41.938755 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3f410-3f91-4fd9-a2b4-30e568bd5f76 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:41.939034 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3f410-3f91-4fd9-a2b4-30e568bd5f76 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:41.939360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3f410-3f91-4fd9-a2b4-30e568bd5f76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:41.948386 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:41.952204 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3f410-3f91-4fd9-a2b4-30e568bd5f76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:41.957853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-e8d3f410-3f91-4fd9-a2b4-30e568bd5f76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:41.957853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:41.957853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:41.957853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:41.957853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:41.957853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:41.957853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:41.957853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:41.957853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:41.957853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:41.957853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:41.957853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:41.957853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:41.957853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:41.957853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:41.957853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:41.966966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:41.966966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:41.966966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:41.966966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:41.966966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:41.966966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:41.966966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:41.966966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:41.966966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:41.966966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:41.966966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:41.966966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:41.966966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:41.966966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:41.966966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:41.966966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:41.968450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:41.968450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:41.968450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:41.968450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:41.968450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:41.968450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:41.968450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:41.968450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:41.968450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:41.968450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:41.968450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:41.968450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:41.968450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 95/191] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:37:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:41.968450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3f410-3f91-4fd9-a2b4-30e568bd5f76 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:41.971904 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d52b03-a0d2-4d46-9141-2c0be1df9bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:41.971904 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d52b03-a0d2-4d46-9141-2c0be1df9bfd None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:41.971904 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d52b03-a0d2-4d46-9141-2c0be1df9bfd None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:41.971904 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d52b03-a0d2-4d46-9141-2c0be1df9bfd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:41.983872 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d52b03-a0d2-4d46-9141-2c0be1df9bfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:41.988834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-d6d52b03-a0d2-4d46-9141-2c0be1df9bfd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:37:41.988834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:41.988834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:41.988834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:41.988834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:41.988834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:41.988834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:41.988834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:41.988834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:41.988834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:41.988834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:41.988834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:41.988834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:41.988834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:41.988834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:41.988834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:41.989712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:41.989712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:41.989712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:41.989712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:41.989712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:41.989712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:41.989712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:41.989712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:41.989712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:41.989712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:41.989712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:41.989712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:41.989712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:41.989712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:41.989712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:41.989712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:41.990496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:41.990496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:41.990496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:41.990496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:41.990496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:41.990496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:41.990496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:41.990496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:41.990496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:41.990496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:41.990496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:37:41.990496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:37:41.990496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 97/192] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:37:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:41.991179 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d52b03-a0d2-4d46-9141-2c0be1df9bfd None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:42.068987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3f410-3f91-4fd9-a2b4-30e568bd5f76 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:42.082901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:42.097390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d52b03-a0d2-4d46-9141-2c0be1df9bfd None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:42.101944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70fc10-15ba-4e19-93bc-ba215569b34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:42.102562 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70fc10-15ba-4e19-93bc-ba215569b34b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:42.102835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70fc10-15ba-4e19-93bc-ba215569b34b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:42.103126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70fc10-15ba-4e19-93bc-ba215569b34b None admin] PATH_INFO: `/v3/users/ce5c8b94724b444f90b8cf10caac2c1a` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:42.104242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70fc10-15ba-4e19-93bc-ba215569b34b None admin] RBAC: Authorizing `identity:get_user(user_id=ce5c8b94724b444f90b8cf10caac2c1a)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:42.105699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70fc10-15ba-4e19-93bc-ba215569b34b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:42.107286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 96/193] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:41 2022] GET /identity/v3/users/ce5c8b94724b444f90b8cf10caac2c1a => generated 289 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:42.108865 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70fc10-15ba-4e19-93bc-ba215569b34b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:42.130448 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723de857-50d9-4b47-82ce-5e4e7e48e7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:42.131129 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723de857-50d9-4b47-82ce-5e4e7e48e7fa None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:42.131412 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723de857-50d9-4b47-82ce-5e4e7e48e7fa None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:42.131690 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723de857-50d9-4b47-82ce-5e4e7e48e7fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:42.132787 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723de857-50d9-4b47-82ce-5e4e7e48e7fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:42.134225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723de857-50d9-4b47-82ce-5e4e7e48e7fa None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:42.135813 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 98/194] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:42.136556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723de857-50d9-4b47-82ce-5e4e7e48e7fa None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:42.201170 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:42.206452 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70fc10-15ba-4e19-93bc-ba215569b34b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:42.231399 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723de857-50d9-4b47-82ce-5e4e7e48e7fa None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:42.241845 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64add196-833e-4fa4-8eb4-c667a25b2f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:42.242497 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64add196-833e-4fa4-8eb4-c667a25b2f9f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:42.242777 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64add196-833e-4fa4-8eb4-c667a25b2f9f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:42.243052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64add196-833e-4fa4-8eb4-c667a25b2f9f None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:42.243764 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64add196-833e-4fa4-8eb4-c667a25b2f9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:42.244062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64add196-833e-4fa4-8eb4-c667a25b2f9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:42.245483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64add196-833e-4fa4-8eb4-c667a25b2f9f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:42.258610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 97/195] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:42.258610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64add196-833e-4fa4-8eb4-c667a25b2f9f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:42.268179 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75d1b6-a53c-4bce-a691-f99ea4971659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:42.268813 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75d1b6-a53c-4bce-a691-f99ea4971659 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:42.269089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75d1b6-a53c-4bce-a691-f99ea4971659 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:42.269359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75d1b6-a53c-4bce-a691-f99ea4971659 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:42.270019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75d1b6-a53c-4bce-a691-f99ea4971659 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:42.270325 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75d1b6-a53c-4bce-a691-f99ea4971659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:42.271722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75d1b6-a53c-4bce-a691-f99ea4971659 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:42.279949 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 99/196] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:37:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:42.280703 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75d1b6-a53c-4bce-a691-f99ea4971659 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:42.298428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:42.362706 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64add196-833e-4fa4-8eb4-c667a25b2f9f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:42.378710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75d1b6-a53c-4bce-a691-f99ea4971659 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:42.390526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:42.396107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a5b44-755e-4ee5-9ed1-914d3ba3ba77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:42.396736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a5b44-755e-4ee5-9ed1-914d3ba3ba77 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:42.397017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a5b44-755e-4ee5-9ed1-914d3ba3ba77 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:42.397289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a5b44-755e-4ee5-9ed1-914d3ba3ba77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:42.412364 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ce900-8380-4595-badf-bbe5c8c80dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:42.412955 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ce900-8380-4595-badf-bbe5c8c80dae None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:42.413230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ce900-8380-4595-badf-bbe5c8c80dae None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:42.413496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ce900-8380-4595-badf-bbe5c8c80dae None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:42.414226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ce900-8380-4595-badf-bbe5c8c80dae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:42.415689 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ce900-8380-4595-badf-bbe5c8c80dae None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:42.464304 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a5b44-755e-4ee5-9ed1-914d3ba3ba77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:42.465784 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a5b44-755e-4ee5-9ed1-914d3ba3ba77 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:42.531410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 100/197] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:37:42 2022] POST /identity/v3/users => generated 297 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:37:42.532313 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ce900-8380-4595-badf-bbe5c8c80dae None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:42.581548 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 98/198] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:42.582531 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a5b44-755e-4ee5-9ed1-914d3ba3ba77 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:42.614399 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:42.622746 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:42.647945 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ce900-8380-4595-badf-bbe5c8c80dae None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:42.691116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa41b24-da41-40e6-bc4e-b056b954deda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:42.691901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa41b24-da41-40e6-bc4e-b056b954deda None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:42.692923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa41b24-da41-40e6-bc4e-b056b954deda None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:42.692923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa41b24-da41-40e6-bc4e-b056b954deda None admin] PATH_INFO: `/v3/users/e4a5235939b84f75b1faa03789462d79` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:42.701534 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a5b44-755e-4ee5-9ed1-914d3ba3ba77 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:42.719178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa41b24-da41-40e6-bc4e-b056b954deda None admin] RBAC: Authorizing `identity:get_user(user_id=e4a5235939b84f75b1faa03789462d79)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:42.720613 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa41b24-da41-40e6-bc4e-b056b954deda None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:42.722453 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 101/199] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:42 2022] GET /identity/v3/users/e4a5235939b84f75b1faa03789462d79 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:42.727726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa41b24-da41-40e6-bc4e-b056b954deda None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:42.744926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2afeef-2514-4b2b-8183-e91ad0d0b1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:42.745564 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2afeef-2514-4b2b-8183-e91ad0d0b1c0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:42.745840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2afeef-2514-4b2b-8183-e91ad0d0b1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:42.746352 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2afeef-2514-4b2b-8183-e91ad0d0b1c0 None admin] PATH_INFO: `/v3/users/e4a5235939b84f75b1faa03789462d79` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:42.747350 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2afeef-2514-4b2b-8183-e91ad0d0b1c0 None admin] RBAC: Authorizing `identity:get_user(user_id=e4a5235939b84f75b1faa03789462d79)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:42.749413 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2afeef-2514-4b2b-8183-e91ad0d0b1c0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:42.751165 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 99/200] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:42 2022] GET /identity/v3/users/e4a5235939b84f75b1faa03789462d79 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:42.756042 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2afeef-2514-4b2b-8183-e91ad0d0b1c0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:42.855813 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa41b24-da41-40e6-bc4e-b056b954deda None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:42.887201 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:42.887201 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2afeef-2514-4b2b-8183-e91ad0d0b1c0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:42.900205 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:42.908170 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410cba5-d32a-40dd-9341-81cd0abe7a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:42.909278 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410cba5-d32a-40dd-9341-81cd0abe7a00 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:42.909560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410cba5-d32a-40dd-9341-81cd0abe7a00 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:42.910119 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410cba5-d32a-40dd-9341-81cd0abe7a00 None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:42.911445 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410cba5-d32a-40dd-9341-81cd0abe7a00 None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:42.913598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410cba5-d32a-40dd-9341-81cd0abe7a00 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:42.916385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 102/201] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:42 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:42.917778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410cba5-d32a-40dd-9341-81cd0abe7a00 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:42.930285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698950d-317e-4e0e-a206-2828839f71ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:42.934491 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698950d-317e-4e0e-a206-2828839f71ac None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:42.934491 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698950d-317e-4e0e-a206-2828839f71ac None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:42.934491 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698950d-317e-4e0e-a206-2828839f71ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:43.011754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698950d-317e-4e0e-a206-2828839f71ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:43.013241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698950d-317e-4e0e-a206-2828839f71ac None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:43.055520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410cba5-d32a-40dd-9341-81cd0abe7a00 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:43.097055 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0703e-8b25-4d6a-989b-12703a7455d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:43.097717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0703e-8b25-4d6a-989b-12703a7455d2 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:43.097992 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0703e-8b25-4d6a-989b-12703a7455d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:43.098260 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0703e-8b25-4d6a-989b-12703a7455d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:43.126717 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:43.135193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 100/202] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:43.140380 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698950d-317e-4e0e-a206-2828839f71ac None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:43.175510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:43.180003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0703e-8b25-4d6a-989b-12703a7455d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:43.181481 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0703e-8b25-4d6a-989b-12703a7455d2 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:43.263726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698950d-317e-4e0e-a206-2828839f71ac None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:43.295567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 103/203] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:43.296974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0703e-8b25-4d6a-989b-12703a7455d2 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:43.303351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce4038-8858-4a20-a54f-c859f5c7b40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:43.303962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce4038-8858-4a20-a54f-c859f5c7b40e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:43.304237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce4038-8858-4a20-a54f-c859f5c7b40e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:43.304504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce4038-8858-4a20-a54f-c859f5c7b40e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:43.305236 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce4038-8858-4a20-a54f-c859f5c7b40e None admin] RBAC: Adding query filter params (role.id=486acd6fe1174148b9461c56c9d371aa, scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4, user.id=ce5c8b94724b444f90b8cf10caac2c1a) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:43.305540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce4038-8858-4a20-a54f-c859f5c7b40e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:43.307071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce4038-8858-4a20-a54f-c859f5c7b40e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:43.323151 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 101/204] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:43 2022] GET /identity/v3/role_assignments?user.id=ce5c8b94724b444f90b8cf10caac2c1a&scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4&role.id=486acd6fe1174148b9461c56c9d371aa => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:43.323151 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce4038-8858-4a20-a54f-c859f5c7b40e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:43.389772 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:43.453144 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce4038-8858-4a20-a54f-c859f5c7b40e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:43.501799 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:43.507924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0703e-8b25-4d6a-989b-12703a7455d2 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:43.521449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b39a2e-d459-4880-b7de-b32aba8062d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:43.522482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b39a2e-d459-4880-b7de-b32aba8062d4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:43.522965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b39a2e-d459-4880-b7de-b32aba8062d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:43.523453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b39a2e-d459-4880-b7de-b32aba8062d4 None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:43.528094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b39a2e-d459-4880-b7de-b32aba8062d4 None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:43.530098 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b39a2e-d459-4880-b7de-b32aba8062d4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:43.532406 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 102/205] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:43 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:43.537883 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b39a2e-d459-4880-b7de-b32aba8062d4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:43.545612 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1a9b7-1ed3-4345-a652-7e67f23ba8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:43.546223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1a9b7-1ed3-4345-a652-7e67f23ba8ec None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:43.546498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1a9b7-1ed3-4345-a652-7e67f23ba8ec None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:43.546766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1a9b7-1ed3-4345-a652-7e67f23ba8ec None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:43.548101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1a9b7-1ed3-4345-a652-7e67f23ba8ec None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:43.549558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1a9b7-1ed3-4345-a652-7e67f23ba8ec None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:43.551221 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 104/206] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:43 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:43.557970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1a9b7-1ed3-4345-a652-7e67f23ba8ec None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:43.681514 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b39a2e-d459-4880-b7de-b32aba8062d4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:43.688185 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1a9b7-1ed3-4345-a652-7e67f23ba8ec None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:43.730881 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:43.734319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adba11a-cc2a-4c1d-9048-49b046b59d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:43.735361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adba11a-cc2a-4c1d-9048-49b046b59d27 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:43.735928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adba11a-cc2a-4c1d-9048-49b046b59d27 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:43.736380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adba11a-cc2a-4c1d-9048-49b046b59d27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:43.737285 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adba11a-cc2a-4c1d-9048-49b046b59d27 None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8, user.id=e4a5235939b84f75b1faa03789462d79) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:43.737763 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adba11a-cc2a-4c1d-9048-49b046b59d27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:43.739726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adba11a-cc2a-4c1d-9048-49b046b59d27 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:43.740746 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5151cf2-fef1-4688-b35c-1e924c2f784a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:43.741293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5151cf2-fef1-4688-b35c-1e924c2f784a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:43.741636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5151cf2-fef1-4688-b35c-1e924c2f784a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:43.741972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5151cf2-fef1-4688-b35c-1e924c2f784a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:43.744526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5151cf2-fef1-4688-b35c-1e924c2f784a None admin] RBAC: Adding query filter params (role.id=6fd5089acea448219d4c0a9a9424a3af, scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8, user.id=e4a5235939b84f75b1faa03789462d79) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:43.744526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5151cf2-fef1-4688-b35c-1e924c2f784a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:43.744526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5151cf2-fef1-4688-b35c-1e924c2f784a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:43.754824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 103/207] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:43 2022] GET /identity/v3/role_assignments?user.id=e4a5235939b84f75b1faa03789462d79&scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8&role.id=6fd5089acea448219d4c0a9a9424a3af => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:43.757960 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 105/208] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:43 2022] GET /identity/v3/role_assignments?user.id=e4a5235939b84f75b1faa03789462d79&scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:44.661530 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea1e08-0e93-481d-b004-db5dd9acd287 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:44.661859 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea1e08-0e93-481d-b004-db5dd9acd287 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:44.662076 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea1e08-0e93-481d-b004-db5dd9acd287 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:44.663717 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 104/209] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:44.668726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2daaa-ff9d-492e-b7da-0a8335fa693f None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:44.669012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2daaa-ff9d-492e-b7da-0a8335fa693f None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:44.669289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2daaa-ff9d-492e-b7da-0a8335fa693f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:44.679747 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cd86c-7832-4ec8-b7a4-5104d27edf99 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:44.680029 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cd86c-7832-4ec8-b7a4-5104d27edf99 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:44.680303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cd86c-7832-4ec8-b7a4-5104d27edf99 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:44.681613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 105/210] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:37:44.688506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b7cc0-61cc-49ef-8bdc-376c297f22f5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:44.688953 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b7cc0-61cc-49ef-8bdc-376c297f22f5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:44.689450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b7cc0-61cc-49ef-8bdc-376c297f22f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:44.689563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-7fa2daaa-ff9d-492e-b7da-0a8335fa693f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:44.690748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7fa2daaa-ff9d-492e-b7da-0a8335fa693f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:44.708922 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-9c8b7cc0-61cc-49ef-8bdc-376c297f22f5 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:44.710370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9c8b7cc0-61cc-49ef-8bdc-376c297f22f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:44.715622 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:44.759533 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 106/211] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:44.782473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 106/212] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:37:45.027071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2daaa-ff9d-492e-b7da-0a8335fa693f None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:45.053567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b7cc0-61cc-49ef-8bdc-376c297f22f5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:45.127315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2daaa-ff9d-492e-b7da-0a8335fa693f None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:45.156576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b7cc0-61cc-49ef-8bdc-376c297f22f5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:45.163252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ff0db-91d4-4369-8233-943cdd4fc988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:45.163880 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ff0db-91d4-4369-8233-943cdd4fc988 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:45.164153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ff0db-91d4-4369-8233-943cdd4fc988 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:45.164418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ff0db-91d4-4369-8233-943cdd4fc988 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:45.165613 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ff0db-91d4-4369-8233-943cdd4fc988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:45.166958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ff0db-91d4-4369-8233-943cdd4fc988 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:45.173083 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 107/213] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:45.174346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ff0db-91d4-4369-8233-943cdd4fc988 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:45.182993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:45.194648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f5f4a-296d-4647-ac19-0e67bcb7fb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:45.202259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f5f4a-296d-4647-ac19-0e67bcb7fb27 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:45.202259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f5f4a-296d-4647-ac19-0e67bcb7fb27 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:45.202259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f5f4a-296d-4647-ac19-0e67bcb7fb27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:45.207501 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:45.207958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f5f4a-296d-4647-ac19-0e67bcb7fb27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:45.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-374f5f4a-296d-4647-ac19-0e67bcb7fb27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:45.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:45.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:45.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:45.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:45.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:45.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:45.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:45.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:45.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:45.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:45.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:45.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:45.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:45.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:45.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:45.214338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:45.214338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:45.214338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:45.214338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:45.214338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:45.214338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:45.214338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:45.214338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:45.214338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:45.214338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:45.214338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:45.214338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:45.214338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:45.214338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:45.214338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:45.214338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:45.215134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:45.215134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:45.215134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:45.215134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:45.215134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:45.215134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:45.215134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:45.215134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:45.215134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:45.215134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:45.215134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:45.215134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:45.222494 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 107/214] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:37:45.223847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f5f4a-296d-4647-ac19-0e67bcb7fb27 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:45.283462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ff0db-91d4-4369-8233-943cdd4fc988 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:45.318246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb1456-2bac-4557-9113-4284d0f9e58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:45.318811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb1456-2bac-4557-9113-4284d0f9e58b None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:45.319171 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb1456-2bac-4557-9113-4284d0f9e58b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:45.319398 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb1456-2bac-4557-9113-4284d0f9e58b None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:45.320051 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb1456-2bac-4557-9113-4284d0f9e58b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:45.321475 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb1456-2bac-4557-9113-4284d0f9e58b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:45.336723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f5f4a-296d-4647-ac19-0e67bcb7fb27 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:45.374347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf270f33-7576-43aa-898d-9fd25ea75e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:45.375075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf270f33-7576-43aa-898d-9fd25ea75e42 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:45.375589 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf270f33-7576-43aa-898d-9fd25ea75e42 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:45.375897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf270f33-7576-43aa-898d-9fd25ea75e42 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:45.376634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf270f33-7576-43aa-898d-9fd25ea75e42 None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:45.377018 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf270f33-7576-43aa-898d-9fd25ea75e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:45.378596 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf270f33-7576-43aa-898d-9fd25ea75e42 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:45.387500 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 108/215] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:45.393651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf270f33-7576-43aa-898d-9fd25ea75e42 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:45.408820 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 108/216] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:37:45 2022] POST /identity/v3/users => generated 297 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:37:45.457471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:45.519048 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf270f33-7576-43aa-898d-9fd25ea75e42 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:45.557261 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac9084-c624-4f2f-b38b-f2fb302b5021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:45.557826 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac9084-c624-4f2f-b38b-f2fb302b5021 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:45.558104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac9084-c624-4f2f-b38b-f2fb302b5021 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:45.558382 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac9084-c624-4f2f-b38b-f2fb302b5021 None admin] PATH_INFO: `/v3/users/e3936ed413934f79b1fe2e2dfb8a90d2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:45.573514 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac9084-c624-4f2f-b38b-f2fb302b5021 None admin] RBAC: Authorizing `identity:get_user(user_id=e3936ed413934f79b1fe2e2dfb8a90d2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:45.575066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac9084-c624-4f2f-b38b-f2fb302b5021 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:45.577118 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 109/217] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:45 2022] GET /identity/v3/users/e3936ed413934f79b1fe2e2dfb8a90d2 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:45.582785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb1456-2bac-4557-9113-4284d0f9e58b None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:45.672899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:45.689749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb1456-2bac-4557-9113-4284d0f9e58b None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:45.727197 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87af9f9-07aa-4af9-813a-e22357fd6c4e None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:45.728117 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87af9f9-07aa-4af9-813a-e22357fd6c4e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:45.728603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87af9f9-07aa-4af9-813a-e22357fd6c4e None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:45.728931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f51de-a1ad-4837-b6f7-7c0514f37f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:45.729545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f51de-a1ad-4837-b6f7-7c0514f37f00 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:45.729823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f51de-a1ad-4837-b6f7-7c0514f37f00 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:45.729917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 110/218] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:37:45.730270 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f51de-a1ad-4837-b6f7-7c0514f37f00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:45.736154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8052d-1dfe-4057-af5c-1816fa2c954f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:45.736429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8052d-1dfe-4057-af5c-1816fa2c954f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:45.736700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8052d-1dfe-4057-af5c-1816fa2c954f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:45.761890 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0ae8052d-1dfe-4057-af5c-1816fa2c954f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:45.763089 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0ae8052d-1dfe-4057-af5c-1816fa2c954f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:45.822716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f51de-a1ad-4837-b6f7-7c0514f37f00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:45.824298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f51de-a1ad-4837-b6f7-7c0514f37f00 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:45.853382 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 111/219] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:45.858906 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ab128-48cf-4d7f-9249-035c1ad4321b None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:45.858906 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ab128-48cf-4d7f-9249-035c1ad4321b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:45.859127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ab128-48cf-4d7f-9249-035c1ad4321b None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:45.865419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 112/220] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:45.869395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fe4b7-7fc3-46da-bfe3-7ad77e43de94 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:45.869670 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fe4b7-7fc3-46da-bfe3-7ad77e43de94 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:45.869936 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fe4b7-7fc3-46da-bfe3-7ad77e43de94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:45.906137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-9b6fe4b7-7fc3-46da-bfe3-7ad77e43de94 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:45.907534 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9b6fe4b7-7fc3-46da-bfe3-7ad77e43de94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:45.956291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:45.972799 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:45.990928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 109/221] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:45.996151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f51de-a1ad-4837-b6f7-7c0514f37f00 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:46.027217 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 113/222] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:46.104409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f51de-a1ad-4837-b6f7-7c0514f37f00 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:46.146619 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528ffba-60ed-467f-9c8f-5673c638fb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:46.147281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528ffba-60ed-467f-9c8f-5673c638fb69 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:46.147566 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528ffba-60ed-467f-9c8f-5673c638fb69 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:46.147841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528ffba-60ed-467f-9c8f-5673c638fb69 None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:46.148998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528ffba-60ed-467f-9c8f-5673c638fb69 None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:46.150523 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528ffba-60ed-467f-9c8f-5673c638fb69 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:46.152417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 110/223] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:45 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:46.164192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6fe4b7-7fc3-46da-bfe3-7ad77e43de94 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:46.222744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528ffba-60ed-467f-9c8f-5673c638fb69 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:46.297442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6fe4b7-7fc3-46da-bfe3-7ad77e43de94 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:46.332122 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f0625-0771-4667-a2aa-3bc491d0d521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:46.332731 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f0625-0771-4667-a2aa-3bc491d0d521 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:46.333005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f0625-0771-4667-a2aa-3bc491d0d521 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:46.333274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f0625-0771-4667-a2aa-3bc491d0d521 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:46.334006 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f0625-0771-4667-a2aa-3bc491d0d521 None admin] RBAC: Adding query filter params (role.id=7c4463788f534454b6d8319dbc4d2afc, scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8, user.id=e3936ed413934f79b1fe2e2dfb8a90d2) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:46.334311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f0625-0771-4667-a2aa-3bc491d0d521 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:46.335785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f0625-0771-4667-a2aa-3bc491d0d521 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:46.342500 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 114/224] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:46 2022] GET /identity/v3/role_assignments?user.id=e3936ed413934f79b1fe2e2dfb8a90d2&scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8&role.id=7c4463788f534454b6d8319dbc4d2afc => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:46.361970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:46.386096 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f0625-0771-4667-a2aa-3bc491d0d521 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:46.439497 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:46.464711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528ffba-60ed-467f-9c8f-5673c638fb69 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:46.533295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9295d-fd95-4d7b-9d32-c9bbb2a50647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:46.533940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9295d-fd95-4d7b-9d32-c9bbb2a50647 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:46.534220 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9295d-fd95-4d7b-9d32-c9bbb2a50647 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:46.534492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9295d-fd95-4d7b-9d32-c9bbb2a50647 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:46.554486 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9295d-fd95-4d7b-9d32-c9bbb2a50647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:46.565594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-a6c9295d-fd95-4d7b-9d32-c9bbb2a50647 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:46.565594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:46.565594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:46.565594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:46.565594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:46.565594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:46.565594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:46.565594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:46.565594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:46.565594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:46.565594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:46.565594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:46.565594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:46.565594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:46.565594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:46.565594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:46.566470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:46.566470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:46.566470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:46.566470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:46.566470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:46.566470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:46.566470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:46.566470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:46.566470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:46.566470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:46.566470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:46.566470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:46.566470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:46.566470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:46.566470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:46.566470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:46.567275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:46.567275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:46.567275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:46.567275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:46.567275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:46.567275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:46.567275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:46.567275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:46.567275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:46.567275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:46.567275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:46.567275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:37:46.567275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 111/225] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:37:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:37:46.574021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9295d-fd95-4d7b-9d32-c9bbb2a50647 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:46.588740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f0625-0771-4667-a2aa-3bc491d0d521 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:46.675913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce560206-0b1c-4cff-9216-c979d5828ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:46.676968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce560206-0b1c-4cff-9216-c979d5828ba6 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:46.677450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce560206-0b1c-4cff-9216-c979d5828ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:46.677966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce560206-0b1c-4cff-9216-c979d5828ba6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:46.699950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce560206-0b1c-4cff-9216-c979d5828ba6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:46.711136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9295d-fd95-4d7b-9d32-c9bbb2a50647 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:46.713302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-ce560206-0b1c-4cff-9216-c979d5828ba6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:37:46.713302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:46.713302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:46.713302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:46.713302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:46.713302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:46.713302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:46.713302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:46.713302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:46.713302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:46.713302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:46.713302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:46.713302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:46.713302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:46.713302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:46.713302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:46.714161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:46.714161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:46.714161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:46.714161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:46.714161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:46.714161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:46.714161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:46.714161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:46.714161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:46.714161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:46.714161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:46.714161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:46.714161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:46.714161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:46.714161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:46.714161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:46.714950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:46.714950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:46.714950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:46.714950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:46.714950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:46.714950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:46.714950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:46.714950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:46.714950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:46.714950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:46.714950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:37:46.714950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:46.714950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 115/226] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:37:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:37:46.721879 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce560206-0b1c-4cff-9216-c979d5828ba6 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:46.734450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:46.787457 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7abe60-07b8-42ec-8eb9-d1afd73d6f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:46.788420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7abe60-07b8-42ec-8eb9-d1afd73d6f04 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:46.788901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7abe60-07b8-42ec-8eb9-d1afd73d6f04 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:46.789392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7abe60-07b8-42ec-8eb9-d1afd73d6f04 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:46.790292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7abe60-07b8-42ec-8eb9-d1afd73d6f04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:46.790805 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7abe60-07b8-42ec-8eb9-d1afd73d6f04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:46.792421 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7abe60-07b8-42ec-8eb9-d1afd73d6f04 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:46.817026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 112/227] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:46.828201 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7abe60-07b8-42ec-8eb9-d1afd73d6f04 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:46.850774 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:46.864231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce560206-0b1c-4cff-9216-c979d5828ba6 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:46.906549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd0658-bb10-4c63-b723-5f6f20ecbc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:46.907309 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd0658-bb10-4c63-b723-5f6f20ecbc3f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:46.907588 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd0658-bb10-4c63-b723-5f6f20ecbc3f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:46.907906 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd0658-bb10-4c63-b723-5f6f20ecbc3f None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:46.908639 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd0658-bb10-4c63-b723-5f6f20ecbc3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:46.908942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd0658-bb10-4c63-b723-5f6f20ecbc3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:46.910344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd0658-bb10-4c63-b723-5f6f20ecbc3f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:46.919155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 116/228] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:37:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:46.924596 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd0658-bb10-4c63-b723-5f6f20ecbc3f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:47.036072 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd0658-bb10-4c63-b723-5f6f20ecbc3f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:47.079047 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7abe60-07b8-42ec-8eb9-d1afd73d6f04 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:47.101403 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df46a35-5920-430c-87c8-58c83ab2ddda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:47.102083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df46a35-5920-430c-87c8-58c83ab2ddda None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:47.102364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df46a35-5920-430c-87c8-58c83ab2ddda None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:47.102920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df46a35-5920-430c-87c8-58c83ab2ddda None admin] PATH_INFO: `/v3/users/e4a5235939b84f75b1faa03789462d79` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:47.104895 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df46a35-5920-430c-87c8-58c83ab2ddda None admin] RBAC: Authorizing `identity:get_user(user_id=e4a5235939b84f75b1faa03789462d79)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:47.106347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df46a35-5920-430c-87c8-58c83ab2ddda None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:47.108685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 117/229] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:46 2022] GET /identity/v3/users/e4a5235939b84f75b1faa03789462d79 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:47.119367 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df46a35-5920-430c-87c8-58c83ab2ddda None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:47.152476 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:47.159798 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a420e-a952-4eee-bf56-7c81b5a6fa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:47.160450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a420e-a952-4eee-bf56-7c81b5a6fa76 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:47.160726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a420e-a952-4eee-bf56-7c81b5a6fa76 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:47.160997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a420e-a952-4eee-bf56-7c81b5a6fa76 None admin] PATH_INFO: `/v3/users/e4a5235939b84f75b1faa03789462d79` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:47.162144 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a420e-a952-4eee-bf56-7c81b5a6fa76 None admin] RBAC: Authorizing `identity:get_user(user_id=e4a5235939b84f75b1faa03789462d79)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:47.163605 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a420e-a952-4eee-bf56-7c81b5a6fa76 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:47.165389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 113/230] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:46 2022] GET /identity/v3/users/e4a5235939b84f75b1faa03789462d79 => generated 283 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:47.181839 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a420e-a952-4eee-bf56-7c81b5a6fa76 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:47.240714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df46a35-5920-430c-87c8-58c83ab2ddda None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:47.275237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:47.282232 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315bc0e-5791-48f5-89f0-267a888b2583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:47.282870 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315bc0e-5791-48f5-89f0-267a888b2583 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:47.283234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315bc0e-5791-48f5-89f0-267a888b2583 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:47.283513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315bc0e-5791-48f5-89f0-267a888b2583 None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:47.284759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315bc0e-5791-48f5-89f0-267a888b2583 None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:47.286339 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315bc0e-5791-48f5-89f0-267a888b2583 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:47.288103 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 118/231] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:47 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:47.293716 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315bc0e-5791-48f5-89f0-267a888b2583 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:47.382871 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:47.398558 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315bc0e-5791-48f5-89f0-267a888b2583 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:47.402434 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a420e-a952-4eee-bf56-7c81b5a6fa76 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:47.453823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97f276-f69b-4e9f-be67-ed2be27e0b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:47.454498 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97f276-f69b-4e9f-be67-ed2be27e0b7e None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:47.454788 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97f276-f69b-4e9f-be67-ed2be27e0b7e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:47.455067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97f276-f69b-4e9f-be67-ed2be27e0b7e None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8/users/e4a5235939b84f75b1faa03789462d79/roles/6fd5089acea448219d4c0a9a9424a3af` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:47.457317 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97f276-f69b-4e9f-be67-ed2be27e0b7e None admin] RBAC: Authorizing `identity:create_grant(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8, user_id=e4a5235939b84f75b1faa03789462d79, role_id=6fd5089acea448219d4c0a9a9424a3af)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:47.458818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97f276-f69b-4e9f-be67-ed2be27e0b7e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:47.474873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159665d-239f-45b2-b7a3-c0c127bd5046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:47.477429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159665d-239f-45b2-b7a3-c0c127bd5046 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:47.477429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159665d-239f-45b2-b7a3-c0c127bd5046 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:47.477429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159665d-239f-45b2-b7a3-c0c127bd5046 None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:47.477429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159665d-239f-45b2-b7a3-c0c127bd5046 None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:47.478851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159665d-239f-45b2-b7a3-c0c127bd5046 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:47.480861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 114/232] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:47 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:47.488104 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159665d-239f-45b2-b7a3-c0c127bd5046 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:47.532939 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 119/233] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:37:47 2022] PUT /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8/users/e4a5235939b84f75b1faa03789462d79/roles/6fd5089acea448219d4c0a9a9424a3af => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:37:47.622493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159665d-239f-45b2-b7a3-c0c127bd5046 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:47.634071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:47.663819 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1ffb3-d622-4e07-88dc-b7e47c0de336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:47.664811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1ffb3-d622-4e07-88dc-b7e47c0de336 None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:47.665097 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1ffb3-d622-4e07-88dc-b7e47c0de336 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:47.665373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1ffb3-d622-4e07-88dc-b7e47c0de336 None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8/users/e4a5235939b84f75b1faa03789462d79/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:47.667520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1ffb3-d622-4e07-88dc-b7e47c0de336 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8, user_id=e4a5235939b84f75b1faa03789462d79, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:47.669049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1ffb3-d622-4e07-88dc-b7e47c0de336 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:47.778337 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 115/234] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:37:47 2022] PUT /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8/users/e4a5235939b84f75b1faa03789462d79/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:37:47.793324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8ebca-087b-4185-97e3-023d11ca93de None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:47.793702 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8ebca-087b-4185-97e3-023d11ca93de None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:47.793939 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8ebca-087b-4185-97e3-023d11ca93de None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:47.796027 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 120/235] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:47.802037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241c043-c392-4277-a85a-93bac40bad05 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:47.802329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241c043-c392-4277-a85a-93bac40bad05 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:47.802604 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241c043-c392-4277-a85a-93bac40bad05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:47.827088 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-d241c043-c392-4277-a85a-93bac40bad05 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:47.828682 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d241c043-c392-4277-a85a-93bac40bad05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:47.905749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 116/236] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:47.921408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539810f-9b10-407c-8a00-86eb70780204 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:47.921914 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539810f-9b10-407c-8a00-86eb70780204 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:47.921984 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539810f-9b10-407c-8a00-86eb70780204 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:47.923597 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 121/237] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:47.933300 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ce67b-340a-4643-b03a-f113b859412e None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:47.933857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ce67b-340a-4643-b03a-f113b859412e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:47.934301 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ce67b-340a-4643-b03a-f113b859412e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:47.971614 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-7c8ce67b-340a-4643-b03a-f113b859412e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:47.973164 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7c8ce67b-340a-4643-b03a-f113b859412e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:48.050263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 117/238] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:48.238932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539810f-9b10-407c-8a00-86eb70780204 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:48.327586 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ce67b-340a-4643-b03a-f113b859412e None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:48.379581 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:48.401718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539810f-9b10-407c-8a00-86eb70780204 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:48.460692 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6b7aa-850f-4b24-b24f-7562d6126beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:48.461415 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6b7aa-850f-4b24-b24f-7562d6126beb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:48.461620 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6b7aa-850f-4b24-b24f-7562d6126beb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:48.461890 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6b7aa-850f-4b24-b24f-7562d6126beb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:48.477667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ce67b-340a-4643-b03a-f113b859412e None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:48.492090 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6b7aa-850f-4b24-b24f-7562d6126beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:48.496912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-a3f6b7aa-850f-4b24-b24f-7562d6126beb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:37:48.496912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:48.496912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:48.496912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:48.496912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:48.496912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:48.496912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:48.496912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:48.496912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:48.496912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:48.496912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:48.496912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:48.496912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:48.496912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:48.496912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:48.496912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:48.497756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:48.497756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:48.497756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:48.497756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:48.497756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:48.497756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:48.497756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:48.497756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:48.497756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:48.497756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:48.497756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:48.497756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:48.497756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:48.497756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:48.497756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:48.497756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:48.499066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:48.499066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:48.499066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:48.499066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:48.499066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:48.499066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:48.499066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:48.499066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:48.499066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:48.499066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:48.499066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:37:48.499066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:48.507420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 122/239] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:48.512845 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6b7aa-850f-4b24-b24f-7562d6126beb None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:48.530962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:48.531855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73144af5-8376-45a7-87c9-0824f2fa800d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:48.532467 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73144af5-8376-45a7-87c9-0824f2fa800d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:48.532759 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73144af5-8376-45a7-87c9-0824f2fa800d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:48.533028 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73144af5-8376-45a7-87c9-0824f2fa800d None admin] PATH_INFO: `/v3/domains/default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:48.534802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73144af5-8376-45a7-87c9-0824f2fa800d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:48.536549 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73144af5-8376-45a7-87c9-0824f2fa800d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:48.538399 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 118/240] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:48.543952 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a1111-bbdb-4d1a-91e3-cf360f316c34 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:48.544235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a1111-bbdb-4d1a-91e3-cf360f316c34 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:48.544543 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a1111-bbdb-4d1a-91e3-cf360f316c34 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:48.546345 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 119/241] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:48.550280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a1111-bbdb-4d1a-91e3-cf360f316c34 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:48.636136 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6b7aa-850f-4b24-b24f-7562d6126beb None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:48.663377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:48.670969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a1111-bbdb-4d1a-91e3-cf360f316c34 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:48.680016 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e4f28-b346-41d4-bbb6-9f75e2dd72b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:48.681147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e4f28-b346-41d4-bbb6-9f75e2dd72b7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:48.681434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e4f28-b346-41d4-bbb6-9f75e2dd72b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:48.681708 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e4f28-b346-41d4-bbb6-9f75e2dd72b7 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:48.682396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e4f28-b346-41d4-bbb6-9f75e2dd72b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:48.682719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e4f28-b346-41d4-bbb6-9f75e2dd72b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:48.685214 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e4f28-b346-41d4-bbb6-9f75e2dd72b7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:48.696623 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 123/242] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:48.698313 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2042180-1162-4af3-b8dc-f09cf37a6d1d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:48.698865 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2042180-1162-4af3-b8dc-f09cf37a6d1d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:48.699296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2042180-1162-4af3-b8dc-f09cf37a6d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:48.713510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba736e79-4bdb-4830-8f1d-ef7035100e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:48.714431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba736e79-4bdb-4830-8f1d-ef7035100e46 None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:48.715392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba736e79-4bdb-4830-8f1d-ef7035100e46 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:48.715878 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba736e79-4bdb-4830-8f1d-ef7035100e46 None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:48.717015 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba736e79-4bdb-4830-8f1d-ef7035100e46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:48.718612 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba736e79-4bdb-4830-8f1d-ef7035100e46 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:48.720329 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e2042180-1162-4af3-b8dc-f09cf37a6d1d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:48.721593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e2042180-1162-4af3-b8dc-f09cf37a6d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:48.789631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:48.795498 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 124/243] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:48.799347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2042180-1162-4af3-b8dc-f09cf37a6d1d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:48.821078 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 120/244] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:37:48 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:37:48.910674 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2042180-1162-4af3-b8dc-f09cf37a6d1d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:48.949451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c47402-293c-4868-a3fe-e45b8413a2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:48.950871 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c47402-293c-4868-a3fe-e45b8413a2cb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:48.951408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c47402-293c-4868-a3fe-e45b8413a2cb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:48.951864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c47402-293c-4868-a3fe-e45b8413a2cb None admin] PATH_INFO: `/v3/users/0423bdcd2f33459db20174645345b9bc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:48.967442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c47402-293c-4868-a3fe-e45b8413a2cb None admin] RBAC: Authorizing `identity:get_user(user_id=0423bdcd2f33459db20174645345b9bc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:48.969187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c47402-293c-4868-a3fe-e45b8413a2cb None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:48.971074 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 125/245] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:48 2022] GET /identity/v3/users/0423bdcd2f33459db20174645345b9bc => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:48.977085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba736e79-4bdb-4830-8f1d-ef7035100e46 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:49.103594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba736e79-4bdb-4830-8f1d-ef7035100e46 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:49.138584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c02c5-1f35-4ce7-b9b6-d194631787ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:49.139646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c02c5-1f35-4ce7-b9b6-d194631787ae None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:49.140128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c47402-293c-4868-a3fe-e45b8413a2cb None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:49.140238 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c02c5-1f35-4ce7-b9b6-d194631787ae None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:49.140657 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c02c5-1f35-4ce7-b9b6-d194631787ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:49.175036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:49.217601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:49.231222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c02c5-1f35-4ce7-b9b6-d194631787ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:49.234993 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237c02c5-1f35-4ce7-b9b6-d194631787ae None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:49.283346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c47402-293c-4868-a3fe-e45b8413a2cb None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:49.328125 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd08276-59a8-4e47-9de2-b18d97b3ee33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:49.329121 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd08276-59a8-4e47-9de2-b18d97b3ee33 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:49.329605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd08276-59a8-4e47-9de2-b18d97b3ee33 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:49.330611 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd08276-59a8-4e47-9de2-b18d97b3ee33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:49.344453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd08276-59a8-4e47-9de2-b18d97b3ee33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:49.350247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-5bd08276-59a8-4e47-9de2-b18d97b3ee33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:49.350247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:49.350247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:49.350247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:49.350247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:49.350247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:49.350247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:49.350247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:49.350247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:49.350247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:49.350247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:49.350247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:49.350247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:49.350247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:49.350247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:49.350247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:49.351259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:49.351259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:49.351259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:49.351259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:49.351259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:49.351259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:49.351259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:49.351259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:49.351259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:49.351259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:49.351259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:49.351259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:49.351259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:49.351259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:49.351259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:49.351259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:49.352033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:49.352033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:49.352033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:49.352033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:49.352033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:49.352033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:49.352033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:49.352033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:49.352033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:49.352033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:49.352033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:49.352033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:49.357571 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 126/246] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:37:49.360701 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 121/247] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:49.361033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd08276-59a8-4e47-9de2-b18d97b3ee33 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:49.366165 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c02c5-1f35-4ce7-b9b6-d194631787ae None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:49.435637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:49.486988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd08276-59a8-4e47-9de2-b18d97b3ee33 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:49.502799 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237c02c5-1f35-4ce7-b9b6-d194631787ae None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:49.510032 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:49.531857 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b95e26-a640-4b8c-acb6-727f853c2c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:49.532789 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b95e26-a640-4b8c-acb6-727f853c2c8f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:49.533329 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b95e26-a640-4b8c-acb6-727f853c2c8f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:49.533816 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b95e26-a640-4b8c-acb6-727f853c2c8f None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:49.534721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b95e26-a640-4b8c-acb6-727f853c2c8f None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:49.535278 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b95e26-a640-4b8c-acb6-727f853c2c8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:49.537017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b95e26-a640-4b8c-acb6-727f853c2c8f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:49.547374 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a91f5f-8391-4332-87d4-44d2481b9a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:49.548014 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a91f5f-8391-4332-87d4-44d2481b9a61 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:49.548294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a91f5f-8391-4332-87d4-44d2481b9a61 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:49.548575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a91f5f-8391-4332-87d4-44d2481b9a61 None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:49.549711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a91f5f-8391-4332-87d4-44d2481b9a61 None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:49.552130 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a91f5f-8391-4332-87d4-44d2481b9a61 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:49.553988 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 122/248] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:49 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:49.558427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 127/249] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:49.559902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a91f5f-8391-4332-87d4-44d2481b9a61 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:49.560975 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b95e26-a640-4b8c-acb6-727f853c2c8f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:49.688586 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b95e26-a640-4b8c-acb6-727f853c2c8f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:49.693391 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a91f5f-8391-4332-87d4-44d2481b9a61 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:49.723211 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:49.735559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf240b4-d9e6-4602-91c1-57280f4ed833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:49.735950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970cf05-e847-45b6-ba31-5fb3d362d78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:49.736202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf240b4-d9e6-4602-91c1-57280f4ed833 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:49.739807 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf240b4-d9e6-4602-91c1-57280f4ed833 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:49.739807 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf240b4-d9e6-4602-91c1-57280f4ed833 None admin] PATH_INFO: `/v3/users/e3936ed413934f79b1fe2e2dfb8a90d2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:49.739962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970cf05-e847-45b6-ba31-5fb3d362d78d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:49.739962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970cf05-e847-45b6-ba31-5fb3d362d78d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:49.739962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970cf05-e847-45b6-ba31-5fb3d362d78d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:49.740146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf240b4-d9e6-4602-91c1-57280f4ed833 None admin] RBAC: Authorizing `identity:get_user(user_id=e3936ed413934f79b1fe2e2dfb8a90d2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:49.740222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970cf05-e847-45b6-ba31-5fb3d362d78d None admin] RBAC: Adding query filter params (role.id=486acd6fe1174148b9461c56c9d371aa, scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8, user.id=0423bdcd2f33459db20174645345b9bc) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:49.740222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970cf05-e847-45b6-ba31-5fb3d362d78d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:49.740346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf240b4-d9e6-4602-91c1-57280f4ed833 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:49.745535 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 128/250] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:49 2022] GET /identity/v3/users/e3936ed413934f79b1fe2e2dfb8a90d2 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:49.745694 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970cf05-e847-45b6-ba31-5fb3d362d78d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:49.748661 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf240b4-d9e6-4602-91c1-57280f4ed833 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:49.750982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 123/251] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:49 2022] GET /identity/v3/role_assignments?user.id=0423bdcd2f33459db20174645345b9bc&scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8&role.id=486acd6fe1174148b9461c56c9d371aa => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:49.807647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:49.864532 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce48544-f973-403f-8af1-607b871e5a9c None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:49.865047 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce48544-f973-403f-8af1-607b871e5a9c None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:49.865047 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce48544-f973-403f-8af1-607b871e5a9c None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:49.866393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 124/252] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:37:49.870582 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf240b4-d9e6-4602-91c1-57280f4ed833 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:49.875869 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f605419-10bc-4d6a-b6c1-0d7c252681b4 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:49.876291 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f605419-10bc-4d6a-b6c1-0d7c252681b4 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:49.876674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f605419-10bc-4d6a-b6c1-0d7c252681b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:49.900471 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-1f605419-10bc-4d6a-b6c1-0d7c252681b4 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:49.901890 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1f605419-10bc-4d6a-b6c1-0d7c252681b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:49.937122 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e990598-5fa3-4555-9f9e-a59e78f5c5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:49.938012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e990598-5fa3-4555-9f9e-a59e78f5c5f5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:49.938682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e990598-5fa3-4555-9f9e-a59e78f5c5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:49.939126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e990598-5fa3-4555-9f9e-a59e78f5c5f5 None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:49.940961 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e990598-5fa3-4555-9f9e-a59e78f5c5f5 None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:49.942429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e990598-5fa3-4555-9f9e-a59e78f5c5f5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:49.944116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 129/253] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:49 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:49.949956 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e990598-5fa3-4555-9f9e-a59e78f5c5f5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:50.017784 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 125/254] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:37:50.120399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e990598-5fa3-4555-9f9e-a59e78f5c5f5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:50.135940 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:50.157546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d06f6d-fc79-4c97-8b80-6daefeb4682d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:50.158557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d06f6d-fc79-4c97-8b80-6daefeb4682d None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:50.159118 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d06f6d-fc79-4c97-8b80-6daefeb4682d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:50.159598 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d06f6d-fc79-4c97-8b80-6daefeb4682d None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8/users/e3936ed413934f79b1fe2e2dfb8a90d2/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:50.162094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d06f6d-fc79-4c97-8b80-6daefeb4682d None admin] RBAC: Authorizing `identity:create_grant(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8, user_id=e3936ed413934f79b1fe2e2dfb8a90d2, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:50.163806 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d06f6d-fc79-4c97-8b80-6daefeb4682d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:50.245860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 130/255] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:37:49 2022] PUT /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8/users/e3936ed413934f79b1fe2e2dfb8a90d2/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:37:50.396201 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f605419-10bc-4d6a-b6c1-0d7c252681b4 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:50.506286 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f605419-10bc-4d6a-b6c1-0d7c252681b4 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:50.506652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3c7f4-ee91-432b-a488-c8b43879da32 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:50.506941 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3c7f4-ee91-432b-a488-c8b43879da32 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:50.507256 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3c7f4-ee91-432b-a488-c8b43879da32 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:50.508865 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 131/256] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:50.524956 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:50.528917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29854d9d-c24b-4ae8-a1a4-876336fd6c2a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:50.529147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29854d9d-c24b-4ae8-a1a4-876336fd6c2a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:50.529427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29854d9d-c24b-4ae8-a1a4-876336fd6c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:50.543526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d559f-a9e2-47e5-a343-662a4e6af0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:50.544439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d559f-a9e2-47e5-a343-662a4e6af0ad None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:50.544946 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d559f-a9e2-47e5-a343-662a4e6af0ad None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:50.545383 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d559f-a9e2-47e5-a343-662a4e6af0ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:50.556447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d559f-a9e2-47e5-a343-662a4e6af0ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:50.562145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-443d559f-a9e2-47e5-a343-662a4e6af0ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:37:50.562145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:50.562145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:50.562145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:50.562145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:50.562145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:50.562145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:50.562145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:50.562145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:50.562145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:50.562145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:50.562145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:50.562145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:50.562145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:50.562145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:50.562145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:50.563004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:50.563004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:50.563004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:50.563004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:50.563004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:50.563004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:50.563004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:50.563004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:50.563004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:50.563004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:50.563004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:50.563004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:50.563004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:50.563004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:50.563004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:50.563004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:50.563836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:50.563836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:50.563836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:50.563836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:50.563836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:50.563836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:50.563836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:50.563836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:50.563836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:50.563836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:50.563836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:37:50.563836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:37:50.566031 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 126/257] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:37:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:50.571784 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d559f-a9e2-47e5-a343-662a4e6af0ad None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:50.572027 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-29854d9d-c24b-4ae8-a1a4-876336fd6c2a None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:50.573481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-29854d9d-c24b-4ae8-a1a4-876336fd6c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:50.678259 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d559f-a9e2-47e5-a343-662a4e6af0ad None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:50.714231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee6a3e-1f4c-4106-8760-956bccb1ce72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:50.715117 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee6a3e-1f4c-4106-8760-956bccb1ce72 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:50.715549 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee6a3e-1f4c-4106-8760-956bccb1ce72 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:50.715963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee6a3e-1f4c-4106-8760-956bccb1ce72 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:50.716901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee6a3e-1f4c-4106-8760-956bccb1ce72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:50.717345 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee6a3e-1f4c-4106-8760-956bccb1ce72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:50.718864 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee6a3e-1f4c-4106-8760-956bccb1ce72 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:50.727423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 127/258] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:37:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:50.731329 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 132/259] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:37:50.733280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee6a3e-1f4c-4106-8760-956bccb1ce72 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:50.774699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:50.846838 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee6a3e-1f4c-4106-8760-956bccb1ce72 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:50.893565 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecedfaa-97fc-4d1d-9ef1-f4510a5ae2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:50.894190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecedfaa-97fc-4d1d-9ef1-f4510a5ae2b9 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:50.894443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecedfaa-97fc-4d1d-9ef1-f4510a5ae2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:50.894710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecedfaa-97fc-4d1d-9ef1-f4510a5ae2b9 None admin] PATH_INFO: `/v3/users/e4a5235939b84f75b1faa03789462d79` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:50.899774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecedfaa-97fc-4d1d-9ef1-f4510a5ae2b9 None admin] RBAC: Authorizing `identity:get_user(user_id=e4a5235939b84f75b1faa03789462d79)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:50.901218 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecedfaa-97fc-4d1d-9ef1-f4510a5ae2b9 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:50.903043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 128/260] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:50 2022] GET /identity/v3/users/e4a5235939b84f75b1faa03789462d79 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:50.908712 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29854d9d-c24b-4ae8-a1a4-876336fd6c2a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:51.060883 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29854d9d-c24b-4ae8-a1a4-876336fd6c2a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:51.064147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:51.098847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad287b2-0692-4160-bcc5-c491f63a64c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:51.099488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad287b2-0692-4160-bcc5-c491f63a64c0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:51.099817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad287b2-0692-4160-bcc5-c491f63a64c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:51.100096 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad287b2-0692-4160-bcc5-c491f63a64c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:51.112236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecedfaa-97fc-4d1d-9ef1-f4510a5ae2b9 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:51.172947 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad287b2-0692-4160-bcc5-c491f63a64c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:51.174439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad287b2-0692-4160-bcc5-c491f63a64c0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:51.259870 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecedfaa-97fc-4d1d-9ef1-f4510a5ae2b9 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:51.271460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:51.295293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2a4d4-27a8-4e4b-ab67-a1525657e15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:51.295940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2a4d4-27a8-4e4b-ab67-a1525657e15f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:51.296216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2a4d4-27a8-4e4b-ab67-a1525657e15f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:51.296484 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2a4d4-27a8-4e4b-ab67-a1525657e15f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:51.308443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2a4d4-27a8-4e4b-ab67-a1525657e15f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:51.313932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-52f2a4d4-27a8-4e4b-ab67-a1525657e15f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:51.313932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:51.313932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:51.313932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:51.313932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:51.313932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:51.313932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:51.313932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:51.313932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:51.313932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:51.313932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:51.313932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:51.313932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:51.313932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:51.313932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:51.313932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:51.315007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:51.315007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:51.315007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:51.315007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:51.315007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:51.315007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:51.315007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:51.315007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:51.315007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:51.315007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:51.315007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:51.315007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:51.315007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:51.315007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:51.315007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:51.315007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:51.315851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:51.315851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:51.315851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:51.315851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:51.315851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:51.315851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:51.315851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:51.315851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:51.315851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:51.315851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:51.315851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:37:51.315851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:37:51.315851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 129/261] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:37:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:51.324008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 133/262] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:51.325914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2a4d4-27a8-4e4b-ab67-a1525657e15f None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:51.329782 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad287b2-0692-4160-bcc5-c491f63a64c0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:51.337511 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:51.488539 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2a4d4-27a8-4e4b-ab67-a1525657e15f None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:51.506764 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad287b2-0692-4160-bcc5-c491f63a64c0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:51.523315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ecfd1-58d9-4764-b2b2-4f0ec2575ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:51.523916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ecfd1-58d9-4764-b2b2-4f0ec2575ff1 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:51.524191 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ecfd1-58d9-4764-b2b2-4f0ec2575ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:51.524461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ecfd1-58d9-4764-b2b2-4f0ec2575ff1 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:51.525114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ecfd1-58d9-4764-b2b2-4f0ec2575ff1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:51.525409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ecfd1-58d9-4764-b2b2-4f0ec2575ff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:51.526774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ecfd1-58d9-4764-b2b2-4f0ec2575ff1 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:51.535533 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 130/263] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:51.540851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ecfd1-58d9-4764-b2b2-4f0ec2575ff1 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:51.570185 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:51.590273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce85898-23eb-4e0e-ba92-f42768b95bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:51.591134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce85898-23eb-4e0e-ba92-f42768b95bad None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:51.591609 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce85898-23eb-4e0e-ba92-f42768b95bad None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:51.592114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce85898-23eb-4e0e-ba92-f42768b95bad None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:51.595792 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce85898-23eb-4e0e-ba92-f42768b95bad None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:51.597521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce85898-23eb-4e0e-ba92-f42768b95bad None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:51.599387 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 134/264] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:51 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:51.604980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce85898-23eb-4e0e-ba92-f42768b95bad None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:51.642545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ecfd1-58d9-4764-b2b2-4f0ec2575ff1 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:51.677107 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b398136-9f45-4541-8bab-2f1f907aba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:51.677997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b398136-9f45-4541-8bab-2f1f907aba13 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:51.678449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b398136-9f45-4541-8bab-2f1f907aba13 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:51.678894 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b398136-9f45-4541-8bab-2f1f907aba13 None admin] PATH_INFO: `/v3/users/e4a5235939b84f75b1faa03789462d79` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:51.680238 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b398136-9f45-4541-8bab-2f1f907aba13 None admin] RBAC: Authorizing `identity:get_user(user_id=e4a5235939b84f75b1faa03789462d79)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:51.681923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b398136-9f45-4541-8bab-2f1f907aba13 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:51.684101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 131/265] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:51 2022] GET /identity/v3/users/e4a5235939b84f75b1faa03789462d79 => generated 283 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:51.690448 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b398136-9f45-4541-8bab-2f1f907aba13 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:51.772108 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:51.788167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:51.792604 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b398136-9f45-4541-8bab-2f1f907aba13 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:51.827946 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d63dd8-f780-4b95-8996-a32b99c22e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:51.828871 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d63dd8-f780-4b95-8996-a32b99c22e38 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:51.829341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d63dd8-f780-4b95-8996-a32b99c22e38 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:51.829820 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d63dd8-f780-4b95-8996-a32b99c22e38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:51.835783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce85898-23eb-4e0e-ba92-f42768b95bad None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:51.900159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d63dd8-f780-4b95-8996-a32b99c22e38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:51.902289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d63dd8-f780-4b95-8996-a32b99c22e38 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:51.924492 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64ceeb-d26d-40fd-abac-dc1c4ef9d2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:51.925119 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64ceeb-d26d-40fd-abac-dc1c4ef9d2fe None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:51.925404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64ceeb-d26d-40fd-abac-dc1c4ef9d2fe None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:51.925679 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64ceeb-d26d-40fd-abac-dc1c4ef9d2fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:51.926431 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64ceeb-d26d-40fd-abac-dc1c4ef9d2fe None admin] RBAC: Adding query filter params (role.id=6fd5089acea448219d4c0a9a9424a3af, scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8, user.id=e4a5235939b84f75b1faa03789462d79) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:51.926745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64ceeb-d26d-40fd-abac-dc1c4ef9d2fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:51.928244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64ceeb-d26d-40fd-abac-dc1c4ef9d2fe None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:51.945954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 135/266] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:51 2022] GET /identity/v3/role_assignments?user.id=e4a5235939b84f75b1faa03789462d79&scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8&role.id=6fd5089acea448219d4c0a9a9424a3af => generated 608 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:52.016938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 132/267] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 14:37:52.021998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:52.023510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64ceeb-d26d-40fd-abac-dc1c4ef9d2fe None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:52.213900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64ceeb-d26d-40fd-abac-dc1c4ef9d2fe None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:52.279144 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f63e5-1882-44ee-a754-302c1638c8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:52.279144 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f63e5-1882-44ee-a754-302c1638c8d4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:52.279144 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f63e5-1882-44ee-a754-302c1638c8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:52.279144 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f63e5-1882-44ee-a754-302c1638c8d4 None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:52.279526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f63e5-1882-44ee-a754-302c1638c8d4 None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:52.280482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f63e5-1882-44ee-a754-302c1638c8d4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:52.285084 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 136/268] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:52 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:52.290890 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d63dd8-f780-4b95-8996-a32b99c22e38 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:52.419845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d63dd8-f780-4b95-8996-a32b99c22e38 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:52.455326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4875713-fea2-4c13-8e7e-7f7450366cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:52.456289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4875713-fea2-4c13-8e7e-7f7450366cd1 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:52.456784 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4875713-fea2-4c13-8e7e-7f7450366cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:52.457248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4875713-fea2-4c13-8e7e-7f7450366cd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:52.458173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4875713-fea2-4c13-8e7e-7f7450366cd1 None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8, user.id=e4a5235939b84f75b1faa03789462d79) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:52.458670 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4875713-fea2-4c13-8e7e-7f7450366cd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:52.460310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4875713-fea2-4c13-8e7e-7f7450366cd1 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:52.467324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 133/269] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:52 2022] GET /identity/v3/role_assignments?user.id=e4a5235939b84f75b1faa03789462d79&scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:52.488936 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa2d87-3f52-4b84-af8a-1991c4d0ca47 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:52.489266 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa2d87-3f52-4b84-af8a-1991c4d0ca47 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:52.489496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa2d87-3f52-4b84-af8a-1991c4d0ca47 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:52.490961 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 137/270] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:52.496774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5fe87-9f8c-42fb-8801-06b0abb36710 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:52.497353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5fe87-9f8c-42fb-8801-06b0abb36710 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:52.497852 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5fe87-9f8c-42fb-8801-06b0abb36710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:52.521378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-19d5fe87-9f8c-42fb-8801-06b0abb36710 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:52.522907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-19d5fe87-9f8c-42fb-8801-06b0abb36710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:52.595509 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:52.602901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 134/271] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:52.635917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b08f5-80a1-4a1f-86ae-d735b94f8e72 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:52.636260 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b08f5-80a1-4a1f-86ae-d735b94f8e72 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:52.636473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b08f5-80a1-4a1f-86ae-d735b94f8e72 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:52.637853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 138/272] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:52.644145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353715d-2839-4d06-aada-84bec4a916bd None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:52.644702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353715d-2839-4d06-aada-84bec4a916bd None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:52.645167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353715d-2839-4d06-aada-84bec4a916bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:52.669975 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-b353715d-2839-4d06-aada-84bec4a916bd None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:52.671527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b353715d-2839-4d06-aada-84bec4a916bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:52.760204 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 135/273] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:52.796226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21286c2c-f6bf-4e96-80c3-20bef76a4329 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:52.796516 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21286c2c-f6bf-4e96-80c3-20bef76a4329 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:52.796798 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21286c2c-f6bf-4e96-80c3-20bef76a4329 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:52.798288 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 139/274] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:52.803696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bac4f-a50b-4178-8a55-7f75fbcc3f31 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:52.803982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bac4f-a50b-4178-8a55-7f75fbcc3f31 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:52.804251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bac4f-a50b-4178-8a55-7f75fbcc3f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:52.823265 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-ea3bac4f-a50b-4178-8a55-7f75fbcc3f31 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:52.824448 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ea3bac4f-a50b-4178-8a55-7f75fbcc3f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:52.893732 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 136/275] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:52.920206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21286c2c-f6bf-4e96-80c3-20bef76a4329 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:52.925166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:53.032103 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21286c2c-f6bf-4e96-80c3-20bef76a4329 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:53.043685 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bac4f-a50b-4178-8a55-7f75fbcc3f31 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:53.081718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125402f-2c0e-4be0-94ec-aeed5aa5a139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:53.083344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125402f-2c0e-4be0-94ec-aeed5aa5a139 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:53.083861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125402f-2c0e-4be0-94ec-aeed5aa5a139 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:53.084540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125402f-2c0e-4be0-94ec-aeed5aa5a139 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:53.086418 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125402f-2c0e-4be0-94ec-aeed5aa5a139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:53.088330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125402f-2c0e-4be0-94ec-aeed5aa5a139 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:53.090149 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 140/276] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:53.096948 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125402f-2c0e-4be0-94ec-aeed5aa5a139 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:53.114447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:53.182181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bac4f-a50b-4178-8a55-7f75fbcc3f31 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:53.185848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:53.217498 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125402f-2c0e-4be0-94ec-aeed5aa5a139 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:53.220599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51ea68-97d0-4371-acb7-5adb0e55da52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:53.221237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51ea68-97d0-4371-acb7-5adb0e55da52 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:53.221512 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51ea68-97d0-4371-acb7-5adb0e55da52 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:53.221868 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51ea68-97d0-4371-acb7-5adb0e55da52 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:53.238937 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51ea68-97d0-4371-acb7-5adb0e55da52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:53.246312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-7b51ea68-97d0-4371-acb7-5adb0e55da52 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:37:53.246312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:53.246312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:53.246312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:53.246312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:53.246312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:53.246312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:53.246312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:53.246312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:53.246312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:53.246312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:53.246312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:53.246312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:53.246312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:53.246312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:53.246312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:53.247233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:53.247233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:53.247233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:53.247233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:53.247233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:53.247233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:53.247233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:53.247233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:53.247233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:53.247233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:53.247233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:53.247233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:53.247233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:53.247233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:53.247233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:53.247233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:53.248679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:53.248679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:53.248679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:53.248679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:53.248679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:53.248679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:53.248679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:53.248679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:53.248679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:53.248679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:53.248679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:37:53.248679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:37:53.254214 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 137/277] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:37:53.255305 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51ea68-97d0-4371-acb7-5adb0e55da52 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:53.256088 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e6919-81c8-452b-9f5c-3f1b1d107f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:53.256700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e6919-81c8-452b-9f5c-3f1b1d107f28 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:53.256975 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e6919-81c8-452b-9f5c-3f1b1d107f28 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:53.259771 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e6919-81c8-452b-9f5c-3f1b1d107f28 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:53.263387 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e6919-81c8-452b-9f5c-3f1b1d107f28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:53.266397 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e6919-81c8-452b-9f5c-3f1b1d107f28 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:53.353490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51ea68-97d0-4371-acb7-5adb0e55da52 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:53.363725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:53.384393 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 141/278] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:37:53 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:37:53.384393 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e6919-81c8-452b-9f5c-3f1b1d107f28 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:53.386853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048850c-2b35-4928-a881-437828ee81b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:53.387581 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048850c-2b35-4928-a881-437828ee81b5 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:53.387853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048850c-2b35-4928-a881-437828ee81b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:53.388128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048850c-2b35-4928-a881-437828ee81b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:53.403794 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048850c-2b35-4928-a881-437828ee81b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:53.410147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-6048850c-2b35-4928-a881-437828ee81b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:53.410147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:53.410147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:53.410147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:53.410147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:53.410147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:53.410147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:53.410147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:53.410147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:53.410147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:53.410147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:53.410147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:53.410147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:53.410147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:53.410147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:53.410147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:53.411094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:53.411094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:53.411094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:53.411094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:53.411094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:53.411094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:53.411094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:53.411094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:53.411094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:53.411094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:53.411094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:53.411094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:53.411094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:53.411094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:53.411094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:53.411094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:53.411924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:53.411924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:53.411924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:53.411924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:53.411924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:53.411924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:53.411924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:53.411924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:53.411924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:53.411924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:53.411924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:53.411924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:37:53.411924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 138/279] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:53.413090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c41951-20b4-432c-88d9-5ca758170c10 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:53.413090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c41951-20b4-432c-88d9-5ca758170c10 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:53.413350 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c41951-20b4-432c-88d9-5ca758170c10 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:53.418637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 139/280] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:53.418637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c41951-20b4-432c-88d9-5ca758170c10 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:53.507429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e6919-81c8-452b-9f5c-3f1b1d107f28 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:53.537751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:53.540157 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c41951-20b4-432c-88d9-5ca758170c10 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:53.547889 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2f765-14d5-4181-afed-bbc9825423da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:53.548507 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2f765-14d5-4181-afed-bbc9825423da None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:53.552624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2f765-14d5-4181-afed-bbc9825423da None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:53.552624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2f765-14d5-4181-afed-bbc9825423da None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:53.552624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2f765-14d5-4181-afed-bbc9825423da None admin] RBAC: Adding query filter params (name=reader) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:53.552624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2f765-14d5-4181-afed-bbc9825423da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:53.552624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2f765-14d5-4181-afed-bbc9825423da None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:53.561139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 142/281] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:53.562938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57009d30-0dc9-4f1a-8a5c-a5c31a1928e0 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:53.563372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57009d30-0dc9-4f1a-8a5c-a5c31a1928e0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:53.563779 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57009d30-0dc9-4f1a-8a5c-a5c31a1928e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:53.578459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c6be8-787f-4ce8-b74d-4355296cf4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:53.579071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c6be8-787f-4ce8-b74d-4355296cf4c6 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:53.579387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c6be8-787f-4ce8-b74d-4355296cf4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:53.579661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c6be8-787f-4ce8-b74d-4355296cf4c6 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:53.580337 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c6be8-787f-4ce8-b74d-4355296cf4c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:53.580632 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c6be8-787f-4ce8-b74d-4355296cf4c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:53.582073 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c6be8-787f-4ce8-b74d-4355296cf4c6 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:53.583020 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-57009d30-0dc9-4f1a-8a5c-a5c31a1928e0 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:53.584845 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-57009d30-0dc9-4f1a-8a5c-a5c31a1928e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:53.592085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 140/282] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:53.592913 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c6be8-787f-4ce8-b74d-4355296cf4c6 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:53.634704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:53.673670 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 143/283] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:53.674875 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57009d30-0dc9-4f1a-8a5c-a5c31a1928e0 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:53.739651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c6be8-787f-4ce8-b74d-4355296cf4c6 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:53.782501 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5979aa-06be-4cc0-aa60-a07da4febe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:53.783156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5979aa-06be-4cc0-aa60-a07da4febe07 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:53.783441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5979aa-06be-4cc0-aa60-a07da4febe07 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:53.783716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5979aa-06be-4cc0-aa60-a07da4febe07 None admin] PATH_INFO: `/v3/users/0423bdcd2f33459db20174645345b9bc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:53.784865 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5979aa-06be-4cc0-aa60-a07da4febe07 None admin] RBAC: Authorizing `identity:get_user(user_id=0423bdcd2f33459db20174645345b9bc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:53.786368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5979aa-06be-4cc0-aa60-a07da4febe07 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:53.788188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 141/284] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:53 2022] GET /identity/v3/users/0423bdcd2f33459db20174645345b9bc => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:53.790373 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57009d30-0dc9-4f1a-8a5c-a5c31a1928e0 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:53.793505 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5979aa-06be-4cc0-aa60-a07da4febe07 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:53.813189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:53.828696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd59c5b-8f17-4b17-ba11-d7cfdcb07cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:53.829330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd59c5b-8f17-4b17-ba11-d7cfdcb07cd7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:53.832727 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd59c5b-8f17-4b17-ba11-d7cfdcb07cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:53.832727 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd59c5b-8f17-4b17-ba11-d7cfdcb07cd7 None admin] PATH_INFO: `/v3/users/63d39489025a4bbdaf1674593e008b9a` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:53.845006 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd59c5b-8f17-4b17-ba11-d7cfdcb07cd7 None admin] RBAC: Authorizing `identity:get_user(user_id=63d39489025a4bbdaf1674593e008b9a)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:53.846737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd59c5b-8f17-4b17-ba11-d7cfdcb07cd7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:53.848863 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 144/285] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:53 2022] GET /identity/v3/users/63d39489025a4bbdaf1674593e008b9a => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:53.855638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd59c5b-8f17-4b17-ba11-d7cfdcb07cd7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:53.901292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:53.905233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5979aa-06be-4cc0-aa60-a07da4febe07 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:53.941005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4223b2b-b835-45ab-8ca0-b60c651b03ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:53.941735 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4223b2b-b835-45ab-8ca0-b60c651b03ba None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:53.942083 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4223b2b-b835-45ab-8ca0-b60c651b03ba None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:53.942423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4223b2b-b835-45ab-8ca0-b60c651b03ba None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:53.943702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4223b2b-b835-45ab-8ca0-b60c651b03ba None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:53.945218 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4223b2b-b835-45ab-8ca0-b60c651b03ba None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:53.951756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 142/286] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:53 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:53.953831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4223b2b-b835-45ab-8ca0-b60c651b03ba None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:53.967837 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd59c5b-8f17-4b17-ba11-d7cfdcb07cd7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:54.004737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7132399-cdaa-4eec-8540-0ec77afbbf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:54.005339 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7132399-cdaa-4eec-8540-0ec77afbbf95 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:54.005614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7132399-cdaa-4eec-8540-0ec77afbbf95 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:54.005877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7132399-cdaa-4eec-8540-0ec77afbbf95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:54.006607 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7132399-cdaa-4eec-8540-0ec77afbbf95 None admin] RBAC: Adding query filter params (role.id=7c4463788f534454b6d8319dbc4d2afc, user.id=63d39489025a4bbdaf1674593e008b9a, scope.system=all) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:54.006910 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7132399-cdaa-4eec-8540-0ec77afbbf95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:54.008392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7132399-cdaa-4eec-8540-0ec77afbbf95 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:54.014482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 145/287] 10.222.0.27 () {56 vars in 1308 bytes} [Tue Aug 30 14:37:53 2022] GET /identity/v3/role_assignments?user.id=63d39489025a4bbdaf1674593e008b9a&scope.system=all&role.id=7c4463788f534454b6d8319dbc4d2afc => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:54.071560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4223b2b-b835-45ab-8ca0-b60c651b03ba None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:54.072220 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7132399-cdaa-4eec-8540-0ec77afbbf95 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:54.125198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:54.144488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaba90b-9635-4459-9e39-1a8d28dba28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:54.146172 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaba90b-9635-4459-9e39-1a8d28dba28f None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:54.147449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaba90b-9635-4459-9e39-1a8d28dba28f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:54.148682 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaba90b-9635-4459-9e39-1a8d28dba28f None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8/users/0423bdcd2f33459db20174645345b9bc/roles/486acd6fe1174148b9461c56c9d371aa` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:54.152691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaba90b-9635-4459-9e39-1a8d28dba28f None admin] RBAC: Authorizing `identity:create_grant(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8, user_id=0423bdcd2f33459db20174645345b9bc, role_id=486acd6fe1174148b9461c56c9d371aa)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:54.155027 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaba90b-9635-4459-9e39-1a8d28dba28f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:54.199287 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7132399-cdaa-4eec-8540-0ec77afbbf95 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:54.250560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 143/288] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:37:53 2022] PUT /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8/users/0423bdcd2f33459db20174645345b9bc/roles/486acd6fe1174148b9461c56c9d371aa => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:37:54.253853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309846c-bc07-46db-b362-40ad356d98eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:54.254505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309846c-bc07-46db-b362-40ad356d98eb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:54.254785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309846c-bc07-46db-b362-40ad356d98eb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:54.255182 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309846c-bc07-46db-b362-40ad356d98eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:54.274348 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309846c-bc07-46db-b362-40ad356d98eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:54.279979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-f309846c-bc07-46db-b362-40ad356d98eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:54.279979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:54.279979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:54.279979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:54.279979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:54.279979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:54.279979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:54.279979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:54.279979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:54.279979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:54.279979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:54.279979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:54.279979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:54.279979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:54.279979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:54.279979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:54.284378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:54.284378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:54.284378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:54.284378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:54.284378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:54.284378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:54.284378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:54.284378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:54.284378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:54.284378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:54.284378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:54.284378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:54.284378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:54.284378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:54.284378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:54.284378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:54.285331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:54.285331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:54.285331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:54.285331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:54.285331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:54.285331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:54.285331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:54.285331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:54.285331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:54.285331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:54.285331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:54.285331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:54.285331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 146/289] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:37:54.287288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaba90b-9635-4459-9e39-1a8d28dba28f None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:54.341793 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:54.438641 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaba90b-9635-4459-9e39-1a8d28dba28f None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:54.477705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b718cc-befa-4c02-a488-6b14ee332f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:54.478357 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b718cc-befa-4c02-a488-6b14ee332f04 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:54.478638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b718cc-befa-4c02-a488-6b14ee332f04 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:54.478908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b718cc-befa-4c02-a488-6b14ee332f04 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:54.479597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b718cc-befa-4c02-a488-6b14ee332f04 None admin] RBAC: Adding query filter params (name=member) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:54.479897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b718cc-befa-4c02-a488-6b14ee332f04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:54.481248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b718cc-befa-4c02-a488-6b14ee332f04 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:54.489780 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 144/290] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:54.495027 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309846c-bc07-46db-b362-40ad356d98eb None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:54.612040 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309846c-bc07-46db-b362-40ad356d98eb None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:54.613600 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:54.663918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5490e-addd-4e08-b39d-54e598f15db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:54.664587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5490e-addd-4e08-b39d-54e598f15db9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:54.664868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5490e-addd-4e08-b39d-54e598f15db9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:54.665141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5490e-addd-4e08-b39d-54e598f15db9 None admin] PATH_INFO: `/v3/users/e3936ed413934f79b1fe2e2dfb8a90d2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:54.666468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5490e-addd-4e08-b39d-54e598f15db9 None admin] RBAC: Authorizing `identity:get_user(user_id=e3936ed413934f79b1fe2e2dfb8a90d2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:54.667994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5490e-addd-4e08-b39d-54e598f15db9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:54.670527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 147/291] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:54 2022] GET /identity/v3/users/e3936ed413934f79b1fe2e2dfb8a90d2 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:54.676265 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b718cc-befa-4c02-a488-6b14ee332f04 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:54.805160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:54.815238 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b718cc-befa-4c02-a488-6b14ee332f04 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:54.856166 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f550ae-1fa5-4e63-a9e4-6c75fd04fdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:54.856771 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f550ae-1fa5-4e63-a9e4-6c75fd04fdf9 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:54.857564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f550ae-1fa5-4e63-a9e4-6c75fd04fdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:54.857842 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f550ae-1fa5-4e63-a9e4-6c75fd04fdf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:54.944101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f550ae-1fa5-4e63-a9e4-6c75fd04fdf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:54.945916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f550ae-1fa5-4e63-a9e4-6c75fd04fdf9 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:55.052857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:55.078827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 145/292] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:55.079461 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5490e-addd-4e08-b39d-54e598f15db9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:55.175461 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5490e-addd-4e08-b39d-54e598f15db9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:55.211906 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8eb26-b576-4056-b356-21a1d9f177d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:55.212911 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8eb26-b576-4056-b356-21a1d9f177d5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:55.213445 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8eb26-b576-4056-b356-21a1d9f177d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:55.213947 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8eb26-b576-4056-b356-21a1d9f177d5 None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:55.215316 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8eb26-b576-4056-b356-21a1d9f177d5 None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:55.217493 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8eb26-b576-4056-b356-21a1d9f177d5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:55.219872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 148/293] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:55 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:55.228369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f550ae-1fa5-4e63-a9e4-6c75fd04fdf9 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:55.376537 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f550ae-1fa5-4e63-a9e4-6c75fd04fdf9 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:55.423816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a45124-0666-41a7-8778-ae8af4c793aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:55.424466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a45124-0666-41a7-8778-ae8af4c793aa None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:55.424743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a45124-0666-41a7-8778-ae8af4c793aa None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:55.425010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a45124-0666-41a7-8778-ae8af4c793aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:55.425724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a45124-0666-41a7-8778-ae8af4c793aa None admin] RBAC: Adding query filter params (role.id=7c4463788f534454b6d8319dbc4d2afc, scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8, user.id=e3936ed413934f79b1fe2e2dfb8a90d2) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:55.426025 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a45124-0666-41a7-8778-ae8af4c793aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:55.427459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a45124-0666-41a7-8778-ae8af4c793aa None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:55.435689 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 146/294] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:55 2022] GET /identity/v3/role_assignments?user.id=e3936ed413934f79b1fe2e2dfb8a90d2&scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8&role.id=7c4463788f534454b6d8319dbc4d2afc => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:55.703045 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9dc77-f25f-4a0e-8ea4-7ddf37c3b352 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:55.704022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9dc77-f25f-4a0e-8ea4-7ddf37c3b352 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:55.704503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9dc77-f25f-4a0e-8ea4-7ddf37c3b352 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:55.706056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 149/295] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:55.711559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c27909-31ef-467e-bc3a-400dbbd2a3f0 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:55.711850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c27909-31ef-467e-bc3a-400dbbd2a3f0 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:55.712127 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c27909-31ef-467e-bc3a-400dbbd2a3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:55.732727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e5c27909-31ef-467e-bc3a-400dbbd2a3f0 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:55.734029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e5c27909-31ef-467e-bc3a-400dbbd2a3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:55.786714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:55.804618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 147/296] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:56.084555 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9dc77-f25f-4a0e-8ea4-7ddf37c3b352 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:56.127727 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:56.189576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9dc77-f25f-4a0e-8ea4-7ddf37c3b352 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:56.214972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e30d5a-589f-4d64-a269-4d08aed8577b None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:56.215496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e30d5a-589f-4d64-a269-4d08aed8577b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:56.215604 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e30d5a-589f-4d64-a269-4d08aed8577b None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:56.217739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 148/297] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:56.223160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619e25b-68ae-45e5-bf70-def1d88436cd None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:56.223449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619e25b-68ae-45e5-bf70-def1d88436cd None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:56.223728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619e25b-68ae-45e5-bf70-def1d88436cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:56.227396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed07eea-b8cd-4755-9999-fc91dc1f51f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:56.228295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed07eea-b8cd-4755-9999-fc91dc1f51f8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:56.228773 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed07eea-b8cd-4755-9999-fc91dc1f51f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:56.229240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed07eea-b8cd-4755-9999-fc91dc1f51f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:56.242378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed07eea-b8cd-4755-9999-fc91dc1f51f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:56.248521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-fed07eea-b8cd-4755-9999-fc91dc1f51f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:37:56.248521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:56.248521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:56.248521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:56.248521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:56.248521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:56.248521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:56.248521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:56.248521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:56.248521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:56.248521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:56.248521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:56.248521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:56.248521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:56.248521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:56.248521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:56.249392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:56.249392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:56.249392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:56.249392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:56.249392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:56.249392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:56.249392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:56.249392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:56.249392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:56.249392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:56.249392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:56.249392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:56.249392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:56.249392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:56.249392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:56.249392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:56.250188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:56.250188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:56.250188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:56.250188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:56.250188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:56.250188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:56.250188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:56.250188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:56.250188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:56.250188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:56.250188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:37:56.250188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:56.250780 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e619e25b-68ae-45e5-bf70-def1d88436cd None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:56.250780 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e619e25b-68ae-45e5-bf70-def1d88436cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:56.253220 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 150/298] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:56.259038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed07eea-b8cd-4755-9999-fc91dc1f51f8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:56.329996 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 149/299] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:56.331761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535ee28-f43e-4f85-b081-cdbe0ce12ed9 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:56.332048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535ee28-f43e-4f85-b081-cdbe0ce12ed9 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:56.332319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535ee28-f43e-4f85-b081-cdbe0ce12ed9 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:56.333748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 150/300] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:56.344997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274692f1-adeb-4a8f-921a-e816be1b217c None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:56.345288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274692f1-adeb-4a8f-921a-e816be1b217c None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:56.345568 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274692f1-adeb-4a8f-921a-e816be1b217c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:56.367510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-274692f1-adeb-4a8f-921a-e816be1b217c None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:56.368859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-274692f1-adeb-4a8f-921a-e816be1b217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:56.411520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:56.419298 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed07eea-b8cd-4755-9999-fc91dc1f51f8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:56.448096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 151/301] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:37:56.467349 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67cd76-3701-49a9-ba02-be1e53824564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:56.468255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67cd76-3701-49a9-ba02-be1e53824564 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:56.468729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67cd76-3701-49a9-ba02-be1e53824564 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:56.469249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67cd76-3701-49a9-ba02-be1e53824564 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:56.470107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67cd76-3701-49a9-ba02-be1e53824564 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:56.470649 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67cd76-3701-49a9-ba02-be1e53824564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:56.472273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67cd76-3701-49a9-ba02-be1e53824564 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:56.481324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 151/302] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:56.489312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274692f1-adeb-4a8f-921a-e816be1b217c None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:56.534448 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:56.590992 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274692f1-adeb-4a8f-921a-e816be1b217c None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:56.616628 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67cd76-3701-49a9-ba02-be1e53824564 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:56.626983 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fc9d2-4bd5-4237-8813-1c08013fb83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:56.627696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fc9d2-4bd5-4237-8813-1c08013fb83c None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:56.627973 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fc9d2-4bd5-4237-8813-1c08013fb83c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:56.628242 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fc9d2-4bd5-4237-8813-1c08013fb83c None admin] PATH_INFO: `/v3/users/fd95c5f79dcc4b0ea9246325b82c4d0e` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:56.646386 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fc9d2-4bd5-4237-8813-1c08013fb83c None admin] RBAC: Authorizing `identity:get_user(user_id=fd95c5f79dcc4b0ea9246325b82c4d0e)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:56.648525 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fc9d2-4bd5-4237-8813-1c08013fb83c None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:56.651219 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 152/303] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:56 2022] GET /identity/v3/users/fd95c5f79dcc4b0ea9246325b82c4d0e => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:56.657168 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fc9d2-4bd5-4237-8813-1c08013fb83c None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:56.724903 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67cd76-3701-49a9-ba02-be1e53824564 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:56.759362 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f2d3e-f87b-4e7d-8dca-6e3efd45446e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:56.759983 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f2d3e-f87b-4e7d-8dca-6e3efd45446e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:56.760259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f2d3e-f87b-4e7d-8dca-6e3efd45446e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:56.760526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f2d3e-f87b-4e7d-8dca-6e3efd45446e None admin] PATH_INFO: `/v3/domains/default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:56.760771 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fc9d2-4bd5-4237-8813-1c08013fb83c None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:56.761615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f2d3e-f87b-4e7d-8dca-6e3efd45446e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:56.762282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:56.763056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f2d3e-f87b-4e7d-8dca-6e3efd45446e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:56.765372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 152/304] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:56.765979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f2d3e-f87b-4e7d-8dca-6e3efd45446e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:56.795039 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:56.796264 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b2dad-a25f-47c9-96d3-abb8ae030493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:56.796860 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b2dad-a25f-47c9-96d3-abb8ae030493 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:56.797135 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b2dad-a25f-47c9-96d3-abb8ae030493 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:56.797399 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b2dad-a25f-47c9-96d3-abb8ae030493 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:56.798255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b2dad-a25f-47c9-96d3-abb8ae030493 None admin] RBAC: Adding query filter params (role.id=486acd6fe1174148b9461c56c9d371aa, user.id=fd95c5f79dcc4b0ea9246325b82c4d0e, scope.system=all) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:56.798475 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b2dad-a25f-47c9-96d3-abb8ae030493 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:56.800257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b2dad-a25f-47c9-96d3-abb8ae030493 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:56.805501 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 153/305] 10.222.0.27 () {56 vars in 1308 bytes} [Tue Aug 30 14:37:56 2022] GET /identity/v3/role_assignments?user.id=fd95c5f79dcc4b0ea9246325b82c4d0e&scope.system=all&role.id=486acd6fe1174148b9461c56c9d371aa => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:56.806618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde39043-1d38-419e-8ca4-04079a26438e None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:56.806936 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde39043-1d38-419e-8ca4-04079a26438e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:56.808330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde39043-1d38-419e-8ca4-04079a26438e None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:56.808425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 154/306] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:56.809040 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde39043-1d38-419e-8ca4-04079a26438e None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:56.889222 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f2d3e-f87b-4e7d-8dca-6e3efd45446e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:56.929901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde39043-1d38-419e-8ca4-04079a26438e None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:56.931507 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc30c95-1871-474e-84b5-29a4079b3322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:56.932231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc30c95-1871-474e-84b5-29a4079b3322 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:56.932508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc30c95-1871-474e-84b5-29a4079b3322 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:56.932783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc30c95-1871-474e-84b5-29a4079b3322 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:56.944699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc30c95-1871-474e-84b5-29a4079b3322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:56.949567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-ffc30c95-1871-474e-84b5-29a4079b3322 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:56.949567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:56.949567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:56.949567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:56.949567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:56.949567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:56.949567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:56.949567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:56.949567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:56.949567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:56.949567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:56.949567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:56.949567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:56.949567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:56.949567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:56.949567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:56.950426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:56.950426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:56.950426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:56.950426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:56.950426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:56.950426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:56.950426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:56.950426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:56.950426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:56.950426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:56.950426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:56.950426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:56.950426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:56.950426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:56.950426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:56.950426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:56.951332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:56.951332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:56.951332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:56.951332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:56.951332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:56.951332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:56.951332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:56.951332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:56.951332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:56.951332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:56.951332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:37:56.951332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:56.951332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 153/307] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:56.952046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfde366-513a-4b9c-9b80-937788b9de2d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:56.952311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfde366-513a-4b9c-9b80-937788b9de2d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:56.952590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfde366-513a-4b9c-9b80-937788b9de2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:56.970863 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-8bfde366-513a-4b9c-9b80-937788b9de2d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:56.972081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8bfde366-513a-4b9c-9b80-937788b9de2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:56.973359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fc470-d806-4c8e-a632-ed4d43a2d0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:56.973974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fc470-d806-4c8e-a632-ed4d43a2d0e9 None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:56.974260 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fc470-d806-4c8e-a632-ed4d43a2d0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:56.974529 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fc470-d806-4c8e-a632-ed4d43a2d0e9 None admin] PATH_INFO: `/v3/groups` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:56.975192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fc470-d806-4c8e-a632-ed4d43a2d0e9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:56.976620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fc470-d806-4c8e-a632-ed4d43a2d0e9 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:57.040490 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 154/308] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:57.041338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfde366-513a-4b9c-9b80-937788b9de2d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:57.059140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 155/309] 10.222.0.27 () {60 vars in 1134 bytes} [Tue Aug 30 14:37:56 2022] POST /identity/v3/groups => generated 229 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:37:57.103861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:57.170018 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfde366-513a-4b9c-9b80-937788b9de2d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:57.205835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b3dae-6bf8-4704-a5f0-7435c5265155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:57.206458 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b3dae-6bf8-4704-a5f0-7435c5265155 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:57.206740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b3dae-6bf8-4704-a5f0-7435c5265155 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:57.207017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b3dae-6bf8-4704-a5f0-7435c5265155 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:57.207741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b3dae-6bf8-4704-a5f0-7435c5265155 None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:57.208049 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b3dae-6bf8-4704-a5f0-7435c5265155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:57.209517 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b3dae-6bf8-4704-a5f0-7435c5265155 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:57.219429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 155/310] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:57.225180 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fc470-d806-4c8e-a632-ed4d43a2d0e9 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:57.257829 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:57.316449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b3dae-6bf8-4704-a5f0-7435c5265155 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:57.338352 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fc470-d806-4c8e-a632-ed4d43a2d0e9 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:57.376889 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e04025-494d-4d50-b5c4-a59e24e829b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:57.377944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e04025-494d-4d50-b5c4-a59e24e829b6 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:57.378415 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e04025-494d-4d50-b5c4-a59e24e829b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:57.378876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e04025-494d-4d50-b5c4-a59e24e829b6 None admin] PATH_INFO: `/v3/users/63d39489025a4bbdaf1674593e008b9a` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:57.380343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e04025-494d-4d50-b5c4-a59e24e829b6 None admin] RBAC: Authorizing `identity:get_user(user_id=63d39489025a4bbdaf1674593e008b9a)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:57.382072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e04025-494d-4d50-b5c4-a59e24e829b6 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:57.384361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 156/311] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:57 2022] GET /identity/v3/users/63d39489025a4bbdaf1674593e008b9a => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:57.389494 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e04025-494d-4d50-b5c4-a59e24e829b6 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:57.415421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:57.438998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b3dae-6bf8-4704-a5f0-7435c5265155 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:57.484369 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e61f4-e873-40c0-ad85-bc34f514561d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:57.485355 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e61f4-e873-40c0-ad85-bc34f514561d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:57.485838 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e61f4-e873-40c0-ad85-bc34f514561d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:57.486333 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e61f4-e873-40c0-ad85-bc34f514561d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:57.498499 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e61f4-e873-40c0-ad85-bc34f514561d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:57.504299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-679e61f4-e873-40c0-ad85-bc34f514561d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:37:57.504299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:57.504299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:57.504299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:57.504299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:57.504299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:57.504299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:57.504299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:57.504299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:57.504299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:57.504299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:57.504299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:57.504299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:57.504299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:57.504299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:57.504299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:57.505153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:57.505153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:57.505153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:57.505153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:57.505153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:57.505153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:57.505153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:57.505153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:57.505153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:57.505153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:57.505153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:57.505153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:57.505153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:57.505153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:57.505153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:57.505153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:57.505944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:57.505944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:57.505944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:57.505944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:57.505944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:57.505944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:57.505944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:57.505944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:57.505944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:57.505944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:57.505944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:37:57.505944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:37:57.508599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 156/312] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:37:57.514667 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e61f4-e873-40c0-ad85-bc34f514561d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:57.516467 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:57.526627 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e04025-494d-4d50-b5c4-a59e24e829b6 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:57.572297 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689b430-d56f-4415-b419-15cb51ade48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:57.573083 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689b430-d56f-4415-b419-15cb51ade48d None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:57.573421 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689b430-d56f-4415-b419-15cb51ade48d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:57.573801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689b430-d56f-4415-b419-15cb51ade48d None admin] PATH_INFO: `/v3/system/users/63d39489025a4bbdaf1674593e008b9a/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:57.576249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689b430-d56f-4415-b419-15cb51ade48d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=63d39489025a4bbdaf1674593e008b9a, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:57.577992 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689b430-d56f-4415-b419-15cb51ade48d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:57.628553 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e61f4-e873-40c0-ad85-bc34f514561d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:57.663044 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 157/313] 10.222.0.27 () {58 vars in 1334 bytes} [Tue Aug 30 14:37:57 2022] PUT /identity/v3/system/users/63d39489025a4bbdaf1674593e008b9a/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:37:57.672024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afa86e-e7a5-4116-a18e-2c520453411f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:57.676909 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afa86e-e7a5-4116-a18e-2c520453411f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:57.676909 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afa86e-e7a5-4116-a18e-2c520453411f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:57.676909 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afa86e-e7a5-4116-a18e-2c520453411f None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:57.676909 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afa86e-e7a5-4116-a18e-2c520453411f None admin] RBAC: Adding query filter params (name=reader) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:57.676909 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afa86e-e7a5-4116-a18e-2c520453411f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:57.676909 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afa86e-e7a5-4116-a18e-2c520453411f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:57.687060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 157/314] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:57.692624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689b430-d56f-4415-b419-15cb51ade48d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:57.811435 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689b430-d56f-4415-b419-15cb51ade48d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:57.847031 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de680f9-113f-4669-b1b8-ffb08bf11cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:57.847693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de680f9-113f-4669-b1b8-ffb08bf11cf3 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:57.847971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de680f9-113f-4669-b1b8-ffb08bf11cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:57.848242 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de680f9-113f-4669-b1b8-ffb08bf11cf3 None admin] PATH_INFO: `/v3/users/0423bdcd2f33459db20174645345b9bc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:57.849393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de680f9-113f-4669-b1b8-ffb08bf11cf3 None admin] RBAC: Authorizing `identity:get_user(user_id=0423bdcd2f33459db20174645345b9bc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:57.850836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de680f9-113f-4669-b1b8-ffb08bf11cf3 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:57.852790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 158/315] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:57 2022] GET /identity/v3/users/0423bdcd2f33459db20174645345b9bc => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:57.858759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afa86e-e7a5-4116-a18e-2c520453411f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:57.862033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:57.968968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afa86e-e7a5-4116-a18e-2c520453411f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:58.003867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa92b8-1a95-4213-9499-d63569470517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:58.004858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa92b8-1a95-4213-9499-d63569470517 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:58.005335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa92b8-1a95-4213-9499-d63569470517 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:58.005798 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa92b8-1a95-4213-9499-d63569470517 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:58.076116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa92b8-1a95-4213-9499-d63569470517 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:58.077899 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa92b8-1a95-4213-9499-d63569470517 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:58.079970 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:58.185773 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 158/316] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:37:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:37:58.191125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de680f9-113f-4669-b1b8-ffb08bf11cf3 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:58.220882 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:58.313825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de680f9-113f-4669-b1b8-ffb08bf11cf3 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:58.350960 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadce85a-b880-40c9-9c77-11091b6493ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:58.351621 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadce85a-b880-40c9-9c77-11091b6493ff None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:58.351903 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadce85a-b880-40c9-9c77-11091b6493ff None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:58.352232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadce85a-b880-40c9-9c77-11091b6493ff None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:58.353366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadce85a-b880-40c9-9c77-11091b6493ff None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:58.354798 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadce85a-b880-40c9-9c77-11091b6493ff None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:58.356839 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 159/317] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:37:58 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:58.362682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa92b8-1a95-4213-9499-d63569470517 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:58.469228 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa92b8-1a95-4213-9499-d63569470517 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:58.470736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:58.502998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9daf5e-954d-4c45-9547-96b99b835b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:58.503626 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9daf5e-954d-4c45-9547-96b99b835b4c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:58.503898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9daf5e-954d-4c45-9547-96b99b835b4c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:58.504163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9daf5e-954d-4c45-9547-96b99b835b4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:58.504888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9daf5e-954d-4c45-9547-96b99b835b4c None admin] RBAC: Adding query filter params (role.id=486acd6fe1174148b9461c56c9d371aa, scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8, user.id=0423bdcd2f33459db20174645345b9bc) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:58.505195 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9daf5e-954d-4c45-9547-96b99b835b4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:58.506628 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9daf5e-954d-4c45-9547-96b99b835b4c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:58.512982 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 159/318] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:37:58 2022] GET /identity/v3/role_assignments?user.id=0423bdcd2f33459db20174645345b9bc&scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8&role.id=486acd6fe1174148b9461c56c9d371aa => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:37:58.723478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283d5d6-5f85-46e9-948b-7dd168b1702a None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:58.723758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283d5d6-5f85-46e9-948b-7dd168b1702a None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:58.724032 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283d5d6-5f85-46e9-948b-7dd168b1702a None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:58.725597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 160/319] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:58.730935 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74b4ab-b10d-45dc-a941-49d3040e2252 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:58.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74b4ab-b10d-45dc-a941-49d3040e2252 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:58.731526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74b4ab-b10d-45dc-a941-49d3040e2252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:58.750723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2f74b4ab-b10d-45dc-a941-49d3040e2252 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:58.751946 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2f74b4ab-b10d-45dc-a941-49d3040e2252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:58.823164 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 160/320] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:59.090300 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283d5d6-5f85-46e9-948b-7dd168b1702a None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:59.155023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc27d5f-4dbd-4642-a254-b3701d66352b None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:59.155557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc27d5f-4dbd-4642-a254-b3701d66352b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:59.155658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc27d5f-4dbd-4642-a254-b3701d66352b None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:59.157089 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 161/321] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:37:59.161768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec955911-ef6e-4bb9-9102-d78172178da5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:59.162048 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec955911-ef6e-4bb9-9102-d78172178da5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:59.162317 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec955911-ef6e-4bb9-9102-d78172178da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:59.180407 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ec955911-ef6e-4bb9-9102-d78172178da5 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:59.181583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ec955911-ef6e-4bb9-9102-d78172178da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:59.191303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:59.199748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283d5d6-5f85-46e9-948b-7dd168b1702a None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:59.242078 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66513947-6985-40d7-a82d-f930a2a673da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:59.242716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66513947-6985-40d7-a82d-f930a2a673da None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:59.242990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66513947-6985-40d7-a82d-f930a2a673da None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:59.243295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66513947-6985-40d7-a82d-f930a2a673da None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:59.254990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66513947-6985-40d7-a82d-f930a2a673da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:37:59.259979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-66513947-6985-40d7-a82d-f930a2a673da None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:37:59.259979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:37:59.259979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:37:59.259979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:37:59.259979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:37:59.259979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:37:59.259979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:37:59.259979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:37:59.259979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:37:59.259979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:37:59.259979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:37:59.259979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:37:59.259979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:37:59.259979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:37:59.259979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:37:59.259979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:37:59.268830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:37:59.268830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:37:59.268830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:37:59.268830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:37:59.268830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:37:59.268830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:37:59.268830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:37:59.268830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:37:59.268830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:37:59.268830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:37:59.268830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:37:59.268830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:37:59.268830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:37:59.268830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:37:59.268830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:37:59.268830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:37:59.270250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:37:59.270250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:37:59.270250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:37:59.270250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:37:59.270250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:37:59.270250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:37:59.270250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:37:59.270250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:37:59.270250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:37:59.270250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:37:59.270250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:37:59.270250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:37:59.270250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 161/322] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:37:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:37:59.270250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66513947-6985-40d7-a82d-f930a2a673da None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:59.284429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 162/323] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:37:59.384402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66513947-6985-40d7-a82d-f930a2a673da None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:59.418800 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64d82c-dea7-4545-b86b-3df641cf3132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:59.419995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64d82c-dea7-4545-b86b-3df641cf3132 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:59.420275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64d82c-dea7-4545-b86b-3df641cf3132 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:59.420568 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64d82c-dea7-4545-b86b-3df641cf3132 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:59.421235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64d82c-dea7-4545-b86b-3df641cf3132 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:37:59.421539 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64d82c-dea7-4545-b86b-3df641cf3132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:59.422902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64d82c-dea7-4545-b86b-3df641cf3132 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:59.431196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 162/324] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:37:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:59.436678 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec955911-ef6e-4bb9-9102-d78172178da5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:59.463614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:59.537741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec955911-ef6e-4bb9-9102-d78172178da5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:59.553698 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64d82c-dea7-4545-b86b-3df641cf3132 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:59.575428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:59.575996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6043a5-8462-4ec1-8c02-b674f639090d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:59.576634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6043a5-8462-4ec1-8c02-b674f639090d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:59.576984 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6043a5-8462-4ec1-8c02-b674f639090d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:59.577174 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6043a5-8462-4ec1-8c02-b674f639090d None admin] PATH_INFO: `/v3/users/fd95c5f79dcc4b0ea9246325b82c4d0e` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:59.578298 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6043a5-8462-4ec1-8c02-b674f639090d None admin] RBAC: Authorizing `identity:get_user(user_id=fd95c5f79dcc4b0ea9246325b82c4d0e)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:59.579776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6043a5-8462-4ec1-8c02-b674f639090d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:59.581504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 163/325] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:37:59 2022] GET /identity/v3/users/fd95c5f79dcc4b0ea9246325b82c4d0e => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:59.586866 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6043a5-8462-4ec1-8c02-b674f639090d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:59.667426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64d82c-dea7-4545-b86b-3df641cf3132 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:59.687388 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:59.694241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6043a5-8462-4ec1-8c02-b674f639090d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:59.700435 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5487c74-6299-4518-9ba1-e1c52b59a182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:59.701054 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5487c74-6299-4518-9ba1-e1c52b59a182 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:59.706645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5487c74-6299-4518-9ba1-e1c52b59a182 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:59.706645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5487c74-6299-4518-9ba1-e1c52b59a182 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:59.706645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5487c74-6299-4518-9ba1-e1c52b59a182 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:59.706645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5487c74-6299-4518-9ba1-e1c52b59a182 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:59.706645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 163/326] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:37:59.706995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94ccb5-e660-4091-bb40-e1d250edf07f None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:59.707253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94ccb5-e660-4091-bb40-e1d250edf07f None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:59.707533 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94ccb5-e660-4091-bb40-e1d250edf07f None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:59.708856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 164/327] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:59.711041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94ccb5-e660-4091-bb40-e1d250edf07f None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:59.734909 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef3765-d180-4df1-a282-edbd0c547dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:59.735587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef3765-d180-4df1-a282-edbd0c547dc8 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:59.735874 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef3765-d180-4df1-a282-edbd0c547dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:59.736128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef3765-d180-4df1-a282-edbd0c547dc8 None admin] PATH_INFO: `/v3/system/users/fd95c5f79dcc4b0ea9246325b82c4d0e/roles/486acd6fe1174148b9461c56c9d371aa` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:59.737525 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef3765-d180-4df1-a282-edbd0c547dc8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=fd95c5f79dcc4b0ea9246325b82c4d0e, role_id=486acd6fe1174148b9461c56c9d371aa)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:59.738902 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef3765-d180-4df1-a282-edbd0c547dc8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:59.795815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:59.810789 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 164/328] 10.222.0.27 () {58 vars in 1334 bytes} [Tue Aug 30 14:37:59 2022] PUT /identity/v3/system/users/fd95c5f79dcc4b0ea9246325b82c4d0e/roles/486acd6fe1174148b9461c56c9d371aa => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:37:59.812063 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade9f4d-ca95-46dc-9fb7-fce96af7dd0a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:59.812349 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade9f4d-ca95-46dc-9fb7-fce96af7dd0a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:59.812647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade9f4d-ca95-46dc-9fb7-fce96af7dd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:59.816266 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94ccb5-e660-4091-bb40-e1d250edf07f None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:59.833456 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0ade9f4d-ca95-46dc-9fb7-fce96af7dd0a None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:59.834977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0ade9f4d-ca95-46dc-9fb7-fce96af7dd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:59.855582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd77784-6c27-4ef1-96a0-0ce1a5c73853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:59.856203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd77784-6c27-4ef1-96a0-0ce1a5c73853 None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:59.856507 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd77784-6c27-4ef1-96a0-0ce1a5c73853 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:59.856861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd77784-6c27-4ef1-96a0-0ce1a5c73853 None admin] PATH_INFO: `/v3/groups` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:59.857495 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd77784-6c27-4ef1-96a0-0ce1a5c73853 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:59.858980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd77784-6c27-4ef1-96a0-0ce1a5c73853 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:59.913928 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 165/329] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:37:59.942472 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 165/330] 10.222.0.27 () {60 vars in 1134 bytes} [Tue Aug 30 14:37:59 2022] POST /identity/v3/groups => generated 226 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:38:00.178925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade9f4d-ca95-46dc-9fb7-fce96af7dd0a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:00.277986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:00.290426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade9f4d-ca95-46dc-9fb7-fce96af7dd0a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:00.326718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2d117-2ccd-49d9-850b-dfde115c2fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:00.327398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2d117-2ccd-49d9-850b-dfde115c2fdb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:00.327673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2d117-2ccd-49d9-850b-dfde115c2fdb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:00.327941 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2d117-2ccd-49d9-850b-dfde115c2fdb None admin] PATH_INFO: `/v3/roles/member` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:00.339005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2d117-2ccd-49d9-850b-dfde115c2fdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:00.344038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-0bc2d117-2ccd-49d9-850b-dfde115c2fdb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:38:00.344038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:00.344038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:00.344038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:00.344038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:00.344038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:00.344038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:00.344038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:00.344038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:00.344038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:00.344038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:00.344038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:00.344038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:00.344038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:00.344038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:00.344038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:00.344994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:00.344994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:00.344994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:00.344994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:00.344994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:00.344994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:00.344994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:00.344994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:00.344994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:00.344994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:00.344994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:00.344994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:00.344994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:00.344994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:00.344994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:00.344994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:00.345790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:00.345790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:00.345790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:00.345790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:00.345790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:00.345790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:00.345790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:00.345790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:00.345790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:00.345790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:00.345790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:38:00.345790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:00.347286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 166/331] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:00.351942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd77784-6c27-4ef1-96a0-0ce1a5c73853 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:00.462496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd77784-6c27-4ef1-96a0-0ce1a5c73853 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:00.500905 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a943d4-13ea-47e5-9f56-1fd2b84b8784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:00.501515 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a943d4-13ea-47e5-9f56-1fd2b84b8784 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:00.501787 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a943d4-13ea-47e5-9f56-1fd2b84b8784 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:00.502065 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a943d4-13ea-47e5-9f56-1fd2b84b8784 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:00.502721 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a943d4-13ea-47e5-9f56-1fd2b84b8784 None admin] RBAC: Adding query filter params (name=member) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:00.503017 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a943d4-13ea-47e5-9f56-1fd2b84b8784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:00.504406 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a943d4-13ea-47e5-9f56-1fd2b84b8784 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:00.513673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 166/332] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:38:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:00.522797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2d117-2ccd-49d9-850b-dfde115c2fdb None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:00.715290 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2d117-2ccd-49d9-850b-dfde115c2fdb None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:00.774347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d25ac0-5ca3-49eb-aab1-6a8352045aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:00.775369 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25ac0-5ca3-49eb-aab1-6a8352045aea None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:00.775860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25ac0-5ca3-49eb-aab1-6a8352045aea None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:00.776395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25ac0-5ca3-49eb-aab1-6a8352045aea None admin] PATH_INFO: `/v3/users/63d39489025a4bbdaf1674593e008b9a` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:00.779172 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d25ac0-5ca3-49eb-aab1-6a8352045aea None admin] RBAC: Authorizing `identity:get_user(user_id=63d39489025a4bbdaf1674593e008b9a)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:00.781782 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d25ac0-5ca3-49eb-aab1-6a8352045aea None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:00.791301 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 167/333] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:38:00 2022] GET /identity/v3/users/63d39489025a4bbdaf1674593e008b9a => generated 293 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:00.796203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a943d4-13ea-47e5-9f56-1fd2b84b8784 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:00.809902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:00.945433 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a943d4-13ea-47e5-9f56-1fd2b84b8784 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:00.991144 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a187c-7874-46b2-8230-522fd6e03ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:00.991144 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a187c-7874-46b2-8230-522fd6e03ff0 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:00.991144 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a187c-7874-46b2-8230-522fd6e03ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:00.991144 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a187c-7874-46b2-8230-522fd6e03ff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:00.991951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a187c-7874-46b2-8230-522fd6e03ff0 None admin] RBAC: Adding query filter params (role.id=7c4463788f534454b6d8319dbc4d2afc, user.id=63d39489025a4bbdaf1674593e008b9a, scope.system=all) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:00.992471 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a187c-7874-46b2-8230-522fd6e03ff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:00.994167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a187c-7874-46b2-8230-522fd6e03ff0 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:01.015622 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 167/334] 10.222.0.27 () {56 vars in 1308 bytes} [Tue Aug 30 14:38:00 2022] GET /identity/v3/role_assignments?user.id=63d39489025a4bbdaf1674593e008b9a&scope.system=all&role.id=7c4463788f534454b6d8319dbc4d2afc => generated 510 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:01.894348 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dfa16-9ccc-4009-9c42-400641def03c None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:01.929632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dfa16-9ccc-4009-9c42-400641def03c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:01.929632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dfa16-9ccc-4009-9c42-400641def03c None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:01.929632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 168/335] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:01.930190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23897873-58ee-48f1-a28e-e0ca9b3ee4a7 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:01.930190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23897873-58ee-48f1-a28e-e0ca9b3ee4a7 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:01.930190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23897873-58ee-48f1-a28e-e0ca9b3ee4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:01.930190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-23897873-58ee-48f1-a28e-e0ca9b3ee4a7 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:01.930190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-23897873-58ee-48f1-a28e-e0ca9b3ee4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:02.000388 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:02.014789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 168/336] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:02.066319 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db11ea-741a-4ba0-a301-0c90cfa046ab None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:02.066635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db11ea-741a-4ba0-a301-0c90cfa046ab None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:02.066857 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db11ea-741a-4ba0-a301-0c90cfa046ab None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:02.068538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 169/337] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:02.074083 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb510f2-98ea-42c7-af2c-8cce55affe18 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:02.074634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb510f2-98ea-42c7-af2c-8cce55affe18 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:02.075137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb510f2-98ea-42c7-af2c-8cce55affe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:02.097656 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-4fb510f2-98ea-42c7-af2c-8cce55affe18 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:02.099221 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-4fb510f2-98ea-42c7-af2c-8cce55affe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:02.189827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 169/338] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:02.398295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db11ea-741a-4ba0-a301-0c90cfa046ab None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:02.422171 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:02.518803 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db11ea-741a-4ba0-a301-0c90cfa046ab None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:02.563920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87633872-94fd-4976-9d04-8be5246b1ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:02.564552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87633872-94fd-4976-9d04-8be5246b1ccc None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:02.564826 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87633872-94fd-4976-9d04-8be5246b1ccc None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:02.565090 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87633872-94fd-4976-9d04-8be5246b1ccc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:02.580531 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87633872-94fd-4976-9d04-8be5246b1ccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:02.585392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-87633872-94fd-4976-9d04-8be5246b1ccc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:38:02.585392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:02.585392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:02.585392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:02.585392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:02.585392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:02.585392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:02.585392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:02.585392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:02.585392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:02.585392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:02.585392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:02.585392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:02.585392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:02.585392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:02.585392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:02.586279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:02.586279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:02.586279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:02.586279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:02.586279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:02.586279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:02.586279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:02.586279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:02.586279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:02.586279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:02.586279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:02.586279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:02.586279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:02.586279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:02.586279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:02.586279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:02.587081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:02.587081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:02.587081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:02.587081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:02.587081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:02.587081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:02.587081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:02.587081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:02.587081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:02.587081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:02.587081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:38:02.587081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:02.587081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 170/339] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:02.596212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb510f2-98ea-42c7-af2c-8cce55affe18 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:02.680580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41a65-594c-4e3b-91fd-85a1e5c1480a None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:02.680864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41a65-594c-4e3b-91fd-85a1e5c1480a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:02.681858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41a65-594c-4e3b-91fd-85a1e5c1480a None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:02.682463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 171/340] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:38:02.683226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41a65-594c-4e3b-91fd-85a1e5c1480a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:02.741945 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb510f2-98ea-42c7-af2c-8cce55affe18 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:02.771544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:02.795695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a57ba5-9889-42ca-bb30-c1c6d756320e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:02.796309 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a57ba5-9889-42ca-bb30-c1c6d756320e None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:02.796585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a57ba5-9889-42ca-bb30-c1c6d756320e None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:02.796855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a57ba5-9889-42ca-bb30-c1c6d756320e None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:02.797597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a57ba5-9889-42ca-bb30-c1c6d756320e None admin] RBAC: Adding query filter params (name=reader) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:02.797924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a57ba5-9889-42ca-bb30-c1c6d756320e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:02.802168 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a57ba5-9889-42ca-bb30-c1c6d756320e None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:02.820393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 170/341] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:38:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:02.821671 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b34059-074b-4172-af0c-6bbefd6ffeb9 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:02.821957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b34059-074b-4172-af0c-6bbefd6ffeb9 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:02.822237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b34059-074b-4172-af0c-6bbefd6ffeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:02.854711 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:02.856475 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-99b34059-074b-4172-af0c-6bbefd6ffeb9 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:02.857708 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-99b34059-074b-4172-af0c-6bbefd6ffeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:02.882743 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41a65-594c-4e3b-91fd-85a1e5c1480a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:02.922934 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58180d-17f5-4c13-8ce0-0eef1a93e710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:02.923705 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58180d-17f5-4c13-8ce0-0eef1a93e710 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:02.923885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58180d-17f5-4c13-8ce0-0eef1a93e710 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:02.924140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58180d-17f5-4c13-8ce0-0eef1a93e710 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:02.937669 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58180d-17f5-4c13-8ce0-0eef1a93e710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:02.942617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-2a58180d-17f5-4c13-8ce0-0eef1a93e710 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:38:02.942617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:02.942617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:02.942617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:02.942617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:02.942617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:02.942617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:02.942617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:02.942617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:02.942617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:02.942617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:02.942617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:02.942617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:02.942617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:02.942617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:02.942617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:02.943506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:02.943506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:02.943506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:02.943506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:02.943506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:02.943506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:02.943506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:02.943506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:02.943506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:02.943506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:02.943506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:02.943506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:02.943506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:02.943506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:02.943506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:02.943506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:02.944286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:02.944286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:02.944286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:02.944286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:02.944286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:02.944286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:02.944286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:02.944286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:02.944286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:02.944286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:02.944286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:38:02.944286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:02.944286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 172/342] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:02.944954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58180d-17f5-4c13-8ce0-0eef1a93e710 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:03.017533 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 171/343] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:03.018209 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b34059-074b-4172-af0c-6bbefd6ffeb9 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:03.070580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58180d-17f5-4c13-8ce0-0eef1a93e710 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:03.128497 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4d353-9997-4adb-82de-22bab8240a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:03.129152 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4d353-9997-4adb-82de-22bab8240a42 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:03.129428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4d353-9997-4adb-82de-22bab8240a42 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:03.129698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4d353-9997-4adb-82de-22bab8240a42 None admin] PATH_INFO: `/v3/users/fd95c5f79dcc4b0ea9246325b82c4d0e` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:03.130852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4d353-9997-4adb-82de-22bab8240a42 None admin] RBAC: Authorizing `identity:get_user(user_id=fd95c5f79dcc4b0ea9246325b82c4d0e)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:03.132365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4d353-9997-4adb-82de-22bab8240a42 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:03.139327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 173/344] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:38:02 2022] GET /identity/v3/users/fd95c5f79dcc4b0ea9246325b82c4d0e => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:03.140371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4d353-9997-4adb-82de-22bab8240a42 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:03.156166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:03.169310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b34059-074b-4172-af0c-6bbefd6ffeb9 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:03.196195 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:03.219023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1a390-8edb-474f-b143-ffb093fa9459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:03.219668 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1a390-8edb-474f-b143-ffb093fa9459 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:03.219943 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1a390-8edb-474f-b143-ffb093fa9459 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:03.220208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1a390-8edb-474f-b143-ffb093fa9459 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:03.220881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1a390-8edb-474f-b143-ffb093fa9459 None admin] RBAC: Adding query filter params (name=member) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:03.221183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1a390-8edb-474f-b143-ffb093fa9459 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:03.228434 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1a390-8edb-474f-b143-ffb093fa9459 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:03.247500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 172/345] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:38:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:03.248457 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1a390-8edb-474f-b143-ffb093fa9459 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:03.272863 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4d353-9997-4adb-82de-22bab8240a42 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:03.309502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9438fba-fa5d-418e-923c-dc81cecb20d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:03.310146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9438fba-fa5d-418e-923c-dc81cecb20d5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:03.310426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9438fba-fa5d-418e-923c-dc81cecb20d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:03.310694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9438fba-fa5d-418e-923c-dc81cecb20d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:03.311454 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9438fba-fa5d-418e-923c-dc81cecb20d5 None admin] RBAC: Adding query filter params (role.id=486acd6fe1174148b9461c56c9d371aa, user.id=fd95c5f79dcc4b0ea9246325b82c4d0e, scope.system=all) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:03.311759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9438fba-fa5d-418e-923c-dc81cecb20d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:03.313297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9438fba-fa5d-418e-923c-dc81cecb20d5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:03.328110 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 174/346] 10.222.0.27 () {56 vars in 1308 bytes} [Tue Aug 30 14:38:03 2022] GET /identity/v3/role_assignments?user.id=fd95c5f79dcc4b0ea9246325b82c4d0e&scope.system=all&role.id=486acd6fe1174148b9461c56c9d371aa => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:03.424551 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1a390-8edb-474f-b143-ffb093fa9459 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:03.466782 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d9737-19f1-4c7e-834d-2e7c88fca959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:03.467422 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d9737-19f1-4c7e-834d-2e7c88fca959 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:03.467699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d9737-19f1-4c7e-834d-2e7c88fca959 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:03.467958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d9737-19f1-4c7e-834d-2e7c88fca959 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:03.518178 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339eeb43-9543-431f-8e7a-e079a58a8646 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:03.518505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339eeb43-9543-431f-8e7a-e079a58a8646 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:03.518733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339eeb43-9543-431f-8e7a-e079a58a8646 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:03.520466 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 175/347] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:38:03.526055 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bea09-057a-443a-bd5e-eeb05fc7216c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:03.526352 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bea09-057a-443a-bd5e-eeb05fc7216c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:03.526613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690bea09-057a-443a-bd5e-eeb05fc7216c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:03.526715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:03.560493 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-690bea09-057a-443a-bd5e-eeb05fc7216c None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:03.561721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-690bea09-057a-443a-bd5e-eeb05fc7216c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:03.583436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d9737-19f1-4c7e-834d-2e7c88fca959 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:03.584851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d9737-19f1-4c7e-834d-2e7c88fca959 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:03.643084 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:03.651550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 176/348] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:38:03.652886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bea09-057a-443a-bd5e-eeb05fc7216c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:03.763883 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690bea09-057a-443a-bd5e-eeb05fc7216c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:03.785598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 173/349] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:03.792175 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d9737-19f1-4c7e-834d-2e7c88fca959 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:03.804573 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfb77b-112b-417e-b244-929aa0943a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:03.805215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfb77b-112b-417e-b244-929aa0943a90 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:03.805541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfb77b-112b-417e-b244-929aa0943a90 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:03.805820 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfb77b-112b-417e-b244-929aa0943a90 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:03.817387 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bfb77b-112b-417e-b244-929aa0943a90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:03.822280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-56bfb77b-112b-417e-b244-929aa0943a90 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:38:03.822280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:03.822280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:03.822280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:03.822280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:03.822280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:03.822280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:03.822280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:03.822280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:03.822280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:03.822280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:03.822280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:03.822280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:03.822280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:03.822280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:03.822280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:03.823274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:03.823274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:03.823274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:03.823274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:03.823274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:03.823274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:03.823274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:03.823274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:03.823274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:03.823274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:03.823274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:03.823274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:03.823274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:03.823274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:03.823274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:03.823274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:03.824102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:03.824102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:03.824102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:03.824102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:03.824102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:03.824102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:03.824102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:03.824102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:03.824102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:03.824102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:03.824102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:38:03.824102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:03.826421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 177/350] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:38:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:03.831413 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfb77b-112b-417e-b244-929aa0943a90 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:03.895616 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:03.901074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:03.932331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d9737-19f1-4c7e-834d-2e7c88fca959 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:03.943983 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfb77b-112b-417e-b244-929aa0943a90 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:03.982625 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b384f7-d81a-400e-b341-6d70bc2b72a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:03.983402 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b384f7-d81a-400e-b341-6d70bc2b72a9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:03.983704 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b384f7-d81a-400e-b341-6d70bc2b72a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:03.983983 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b384f7-d81a-400e-b341-6d70bc2b72a9 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:03.984791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b384f7-d81a-400e-b341-6d70bc2b72a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:03.985109 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b384f7-d81a-400e-b341-6d70bc2b72a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:03.986701 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b384f7-d81a-400e-b341-6d70bc2b72a9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:04.001131 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 178/351] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:04.002561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b384f7-d81a-400e-b341-6d70bc2b72a9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:04.014252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48f531-23f5-4db3-a9c6-cc7e486390ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:04.014896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48f531-23f5-4db3-a9c6-cc7e486390ac None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:04.015197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48f531-23f5-4db3-a9c6-cc7e486390ac None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:04.015503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48f531-23f5-4db3-a9c6-cc7e486390ac None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:04.016639 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48f531-23f5-4db3-a9c6-cc7e486390ac None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:04.018086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48f531-23f5-4db3-a9c6-cc7e486390ac None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:04.028384 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 174/352] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:38:03 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:04.034600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48f531-23f5-4db3-a9c6-cc7e486390ac None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:04.179463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b384f7-d81a-400e-b341-6d70bc2b72a9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:04.197412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:04.216075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48f531-23f5-4db3-a9c6-cc7e486390ac None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:04.224911 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f615af-a491-4949-b294-c35838d44e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:04.226207 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f615af-a491-4949-b294-c35838d44e3b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:04.226434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f615af-a491-4949-b294-c35838d44e3b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:04.226708 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f615af-a491-4949-b294-c35838d44e3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:04.272136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075a91d-a441-4483-8d1f-fb4b32088aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:04.272776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075a91d-a441-4483-8d1f-fb4b32088aed None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:04.273054 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075a91d-a441-4483-8d1f-fb4b32088aed None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:04.273327 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075a91d-a441-4483-8d1f-fb4b32088aed None admin] PATH_INFO: `/v3/groups/fb4974a00d9c4982aee946511fde7fee` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:04.280190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075a91d-a441-4483-8d1f-fb4b32088aed None admin] RBAC: Authorizing `identity:get_group(group_id=fb4974a00d9c4982aee946511fde7fee)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:04.281659 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075a91d-a441-4483-8d1f-fb4b32088aed None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:04.283786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 175/353] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 30 14:38:04 2022] GET /identity/v3/groups/fb4974a00d9c4982aee946511fde7fee => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:04.284661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075a91d-a441-4483-8d1f-fb4b32088aed None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:04.304251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:04.334842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f615af-a491-4949-b294-c35838d44e3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:04.336360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f615af-a491-4949-b294-c35838d44e3b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:04.406341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075a91d-a441-4483-8d1f-fb4b32088aed None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:04.444889 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790443c-1882-4808-8a4a-cac52a79b603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:04.445453 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790443c-1882-4808-8a4a-cac52a79b603 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:04.445733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790443c-1882-4808-8a4a-cac52a79b603 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:04.446007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790443c-1882-4808-8a4a-cac52a79b603 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:04.456881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790443c-1882-4808-8a4a-cac52a79b603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:04.457943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 179/354] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:04.458832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f615af-a491-4949-b294-c35838d44e3b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:04.462348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-2790443c-1882-4808-8a4a-cac52a79b603 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:38:04.462348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:04.462348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:04.462348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:04.462348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:04.462348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:04.462348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:04.462348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:04.462348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:04.462348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:04.462348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:04.462348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:04.462348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:04.462348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:04.462348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:04.462348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:04.463231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:04.463231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:04.463231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:04.463231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:04.463231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:04.463231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:04.463231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:04.463231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:04.463231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:04.463231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:04.463231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:04.463231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:04.463231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:04.463231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:04.463231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:04.463231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:04.464026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:04.464026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:04.464026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:04.464026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:04.464026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:04.464026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:04.464026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:04.464026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:04.464026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:04.464026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:04.464026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:38:04.464026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:04.464026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 176/355] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:04.469573 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790443c-1882-4808-8a4a-cac52a79b603 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:04.477233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:04.545315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:04.575689 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f615af-a491-4949-b294-c35838d44e3b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:04.585078 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790443c-1882-4808-8a4a-cac52a79b603 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:04.629709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494f97f-d5b2-4cf1-8944-b5b2ec888f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:04.630433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494f97f-d5b2-4cf1-8944-b5b2ec888f61 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:04.630802 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494f97f-d5b2-4cf1-8944-b5b2ec888f61 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:04.631177 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494f97f-d5b2-4cf1-8944-b5b2ec888f61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:04.631991 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494f97f-d5b2-4cf1-8944-b5b2ec888f61 None admin] RBAC: Adding query filter params (group.id=fb4974a00d9c4982aee946511fde7fee, role.id=7c4463788f534454b6d8319dbc4d2afc, scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:04.632398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494f97f-d5b2-4cf1-8944-b5b2ec888f61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:04.634101 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494f97f-d5b2-4cf1-8944-b5b2ec888f61 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:04.634314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75940828-c34f-4c2b-8d22-844852c98d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:04.635204 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75940828-c34f-4c2b-8d22-844852c98d3f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:04.636793 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75940828-c34f-4c2b-8d22-844852c98d3f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:04.637255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75940828-c34f-4c2b-8d22-844852c98d3f None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:04.638700 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75940828-c34f-4c2b-8d22-844852c98d3f None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:04.640415 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75940828-c34f-4c2b-8d22-844852c98d3f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:04.640593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 180/356] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 30 14:38:04 2022] GET /identity/v3/role_assignments?group.id=fb4974a00d9c4982aee946511fde7fee&scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4&role.id=7c4463788f534454b6d8319dbc4d2afc => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:04.641360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494f97f-d5b2-4cf1-8944-b5b2ec888f61 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:04.642693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 177/357] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:38:04 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:04.644875 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30130197-3795-4bea-8a62-d62450424f81 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:04.645401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30130197-3795-4bea-8a62-d62450424f81 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:04.645905 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30130197-3795-4bea-8a62-d62450424f81 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:04.647627 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 178/358] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:04.654956 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5960f-03fd-4818-b20c-f55534f6d8cc None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:04.655589 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5960f-03fd-4818-b20c-f55534f6d8cc None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:04.656052 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5960f-03fd-4818-b20c-f55534f6d8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:04.676385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-15d5960f-03fd-4818-b20c-f55534f6d8cc None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:04.677991 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-15d5960f-03fd-4818-b20c-f55534f6d8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:04.757630 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 179/359] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:38:04.758931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5960f-03fd-4818-b20c-f55534f6d8cc None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:04.760846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:04.787140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494f97f-d5b2-4cf1-8944-b5b2ec888f61 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:04.830653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8c454-02db-477d-87c0-56799a0ce88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:04.831612 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8c454-02db-477d-87c0-56799a0ce88c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:04.832082 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8c454-02db-477d-87c0-56799a0ce88c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:04.832527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8c454-02db-477d-87c0-56799a0ce88c None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:04.833372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8c454-02db-477d-87c0-56799a0ce88c None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:04.833832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8c454-02db-477d-87c0-56799a0ce88c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:04.835414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8c454-02db-477d-87c0-56799a0ce88c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:04.838140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:04.845494 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 181/360] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:38:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:04.849773 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b683e-0606-4bfe-a666-3216ce345f2b None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:04.849773 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b683e-0606-4bfe-a666-3216ce345f2b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:04.849773 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b683e-0606-4bfe-a666-3216ce345f2b None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:04.849773 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 182/361] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:04.851148 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b683e-0606-4bfe-a666-3216ce345f2b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:04.879170 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5960f-03fd-4818-b20c-f55534f6d8cc None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:04.925242 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb96ab-ec95-44e1-ae3c-b0fac6fadf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:04.926040 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb96ab-ec95-44e1-ae3c-b0fac6fadf6b None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:04.926412 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb96ab-ec95-44e1-ae3c-b0fac6fadf6b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:04.926780 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb96ab-ec95-44e1-ae3c-b0fac6fadf6b None admin] PATH_INFO: `/v3/groups/fb4974a00d9c4982aee946511fde7fee` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:04.928063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb96ab-ec95-44e1-ae3c-b0fac6fadf6b None admin] RBAC: Authorizing `identity:get_group(group_id=fb4974a00d9c4982aee946511fde7fee)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:04.929763 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb96ab-ec95-44e1-ae3c-b0fac6fadf6b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:04.935125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 180/362] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 30 14:38:04 2022] GET /identity/v3/groups/fb4974a00d9c4982aee946511fde7fee => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:04.935125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaf343-f93d-4162-bf58-566d0f07501a None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:04.935125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaf343-f93d-4162-bf58-566d0f07501a None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:04.935125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaf343-f93d-4162-bf58-566d0f07501a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:04.953592 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-0edaf343-f93d-4162-bf58-566d0f07501a None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:04.954912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0edaf343-f93d-4162-bf58-566d0f07501a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:04.998849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b683e-0606-4bfe-a666-3216ce345f2b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:05.031070 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 181/363] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:05.036333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edaf343-f93d-4162-bf58-566d0f07501a None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:05.045345 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add17ca-68a6-4399-8f49-f0e8a74c9b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:05.045956 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add17ca-68a6-4399-8f49-f0e8a74c9b43 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:05.046237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add17ca-68a6-4399-8f49-f0e8a74c9b43 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:05.046565 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add17ca-68a6-4399-8f49-f0e8a74c9b43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:05.058339 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:05.132213 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add17ca-68a6-4399-8f49-f0e8a74c9b43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:05.134049 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add17ca-68a6-4399-8f49-f0e8a74c9b43 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:05.144874 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:05.193578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edaf343-f93d-4162-bf58-566d0f07501a None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:05.242951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae33ad-0d51-4de2-bb7a-19d11461bb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:05.243924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae33ad-0d51-4de2-bb7a-19d11461bb63 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:05.244212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae33ad-0d51-4de2-bb7a-19d11461bb63 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:05.244649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae33ad-0d51-4de2-bb7a-19d11461bb63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:05.245513 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae33ad-0d51-4de2-bb7a-19d11461bb63 None admin] RBAC: Adding query filter params (group.id=fb4974a00d9c4982aee946511fde7fee, role.id=6fd5089acea448219d4c0a9a9424a3af, scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:05.245826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae33ad-0d51-4de2-bb7a-19d11461bb63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:05.248068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae33ad-0d51-4de2-bb7a-19d11461bb63 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:05.255460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 182/364] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 30 14:38:05 2022] GET /identity/v3/role_assignments?group.id=fb4974a00d9c4982aee946511fde7fee&scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4&role.id=6fd5089acea448219d4c0a9a9424a3af => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:05.256263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae33ad-0d51-4de2-bb7a-19d11461bb63 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:05.283324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 183/365] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:05.291632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add17ca-68a6-4399-8f49-f0e8a74c9b43 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:05.359865 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:05.406059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae33ad-0d51-4de2-bb7a-19d11461bb63 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:05.444397 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add17ca-68a6-4399-8f49-f0e8a74c9b43 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:05.451006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:05.460382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4199a18-6cf6-46c2-918f-c61c2180fbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:05.461328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4199a18-6cf6-46c2-918f-c61c2180fbd9 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:05.461611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4199a18-6cf6-46c2-918f-c61c2180fbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:05.461884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4199a18-6cf6-46c2-918f-c61c2180fbd9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:05.477760 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4199a18-6cf6-46c2-918f-c61c2180fbd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:05.483924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-e4199a18-6cf6-46c2-918f-c61c2180fbd9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:38:05.483924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:05.483924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:05.483924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:05.483924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:05.483924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:05.483924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:05.483924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:05.483924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:05.483924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:05.483924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:05.483924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:05.483924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:05.483924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:05.483924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:05.483924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:05.484802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:05.484802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:05.484802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:05.484802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:05.484802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:05.484802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:05.484802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:05.484802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:05.484802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:05.484802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:05.484802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:05.484802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:05.484802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:05.484802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:05.484802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:05.484802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:05.485590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:05.485590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:05.485590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:05.485590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:05.485590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:05.485590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:05.485590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:05.485590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:05.485590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:05.485590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:05.485590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:38:05.485590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:05.485590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 183/366] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:38:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:05.486233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4199a18-6cf6-46c2-918f-c61c2180fbd9 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:05.495772 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a534a65-1506-4117-86eb-f326679ee880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:05.496719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a534a65-1506-4117-86eb-f326679ee880 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:05.497768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a534a65-1506-4117-86eb-f326679ee880 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:05.498239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a534a65-1506-4117-86eb-f326679ee880 None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:05.499708 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a534a65-1506-4117-86eb-f326679ee880 None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:05.501353 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a534a65-1506-4117-86eb-f326679ee880 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:05.503353 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 184/367] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:38:05 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:05.504116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a534a65-1506-4117-86eb-f326679ee880 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:05.616428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4199a18-6cf6-46c2-918f-c61c2180fbd9 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:05.666898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a534a65-1506-4117-86eb-f326679ee880 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:05.668309 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc3514-b330-40ae-8dd0-50d534dfb4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:05.668994 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc3514-b330-40ae-8dd0-50d534dfb4d9 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:05.669275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc3514-b330-40ae-8dd0-50d534dfb4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:05.669547 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc3514-b330-40ae-8dd0-50d534dfb4d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:05.688071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:05.696794 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc3514-b330-40ae-8dd0-50d534dfb4d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:05.707526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-b0cc3514-b330-40ae-8dd0-50d534dfb4d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:05.707526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:05.707526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:05.707526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:05.707526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:05.707526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:05.707526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:05.707526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:05.707526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:05.707526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:05.707526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:05.707526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:05.707526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:05.707526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:05.707526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:05.707526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:05.708479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:05.708479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:05.708479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:05.708479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:05.708479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:05.708479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:05.708479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:05.708479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:05.708479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:05.708479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:05.708479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:05.708479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:05.708479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:05.708479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:05.708479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:05.708479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:05.709263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:05.709263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:05.709263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:05.709263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:05.709263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:05.709263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:05.709263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:05.709263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:05.709263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:05.709263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:05.709263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:05.709263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:05.709263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 184/368] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:38:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:05.710520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc3514-b330-40ae-8dd0-50d534dfb4d9 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:05.739008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bffeea-3282-454a-8c3f-050dbe904723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:05.740516 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bffeea-3282-454a-8c3f-050dbe904723 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:05.741284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bffeea-3282-454a-8c3f-050dbe904723 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:05.741917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bffeea-3282-454a-8c3f-050dbe904723 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:05.743932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bffeea-3282-454a-8c3f-050dbe904723 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:05.744452 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bffeea-3282-454a-8c3f-050dbe904723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:05.746081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bffeea-3282-454a-8c3f-050dbe904723 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:05.759082 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 185/369] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:05.760687 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bffeea-3282-454a-8c3f-050dbe904723 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:05.793167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:05.854873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc3514-b330-40ae-8dd0-50d534dfb4d9 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:05.903806 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bffeea-3282-454a-8c3f-050dbe904723 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:05.903976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f987260-ba36-4c0c-87b2-715b31014287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:05.904096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f987260-ba36-4c0c-87b2-715b31014287 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:05.904379 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f987260-ba36-4c0c-87b2-715b31014287 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:05.904648 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f987260-ba36-4c0c-87b2-715b31014287 None admin] PATH_INFO: `/v3/groups/fb4974a00d9c4982aee946511fde7fee` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:05.906251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f987260-ba36-4c0c-87b2-715b31014287 None admin] RBAC: Authorizing `identity:get_group(group_id=fb4974a00d9c4982aee946511fde7fee)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:05.908754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f987260-ba36-4c0c-87b2-715b31014287 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:05.915262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 185/370] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 30 14:38:05 2022] GET /identity/v3/groups/fb4974a00d9c4982aee946511fde7fee => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:05.916208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f987260-ba36-4c0c-87b2-715b31014287 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:05.950872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169994d-eeae-4d80-935f-f4aaab74f9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:05.952251 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169994d-eeae-4d80-935f-f4aaab74f9a8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:05.952718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169994d-eeae-4d80-935f-f4aaab74f9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:05.953492 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169994d-eeae-4d80-935f-f4aaab74f9a8 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:05.955637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169994d-eeae-4d80-935f-f4aaab74f9a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:05.956226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169994d-eeae-4d80-935f-f4aaab74f9a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:05.958085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169994d-eeae-4d80-935f-f4aaab74f9a8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:05.970180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 186/371] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:05.971721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169994d-eeae-4d80-935f-f4aaab74f9a8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:06.015716 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:06.085693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f987260-ba36-4c0c-87b2-715b31014287 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:06.102857 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169994d-eeae-4d80-935f-f4aaab74f9a8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:06.125062 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:06.149339 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fd19e-08d5-4e47-bbba-6610c1d8cc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:06.150296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fd19e-08d5-4e47-bbba-6610c1d8cc37 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:06.150805 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fd19e-08d5-4e47-bbba-6610c1d8cc37 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:06.151312 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fd19e-08d5-4e47-bbba-6610c1d8cc37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:06.152719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fd19e-08d5-4e47-bbba-6610c1d8cc37 None admin] RBAC: Adding query filter params (group.id=fb4974a00d9c4982aee946511fde7fee, role.id=7c4463788f534454b6d8319dbc4d2afc, scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:06.154135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fd19e-08d5-4e47-bbba-6610c1d8cc37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:06.156058 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fd19e-08d5-4e47-bbba-6610c1d8cc37 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:06.163147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 187/372] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 30 14:38:05 2022] GET /identity/v3/role_assignments?group.id=fb4974a00d9c4982aee946511fde7fee&scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8&role.id=7c4463788f534454b6d8319dbc4d2afc => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:06.164282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fd19e-08d5-4e47-bbba-6610c1d8cc37 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:06.166425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0669e-a7ae-4552-832e-5a36584aa259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:06.167139 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0669e-a7ae-4552-832e-5a36584aa259 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:06.167421 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0669e-a7ae-4552-832e-5a36584aa259 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:06.167700 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0669e-a7ae-4552-832e-5a36584aa259 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:06.345615 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0669e-a7ae-4552-832e-5a36584aa259 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:06.347132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0669e-a7ae-4552-832e-5a36584aa259 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:06.349481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:06.365321 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fd19e-08d5-4e47-bbba-6610c1d8cc37 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:06.436439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2e086-8e8c-4644-bfa5-21ca6f12711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:06.437388 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2e086-8e8c-4644-bfa5-21ca6f12711d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:06.437879 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2e086-8e8c-4644-bfa5-21ca6f12711d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:06.438434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2e086-8e8c-4644-bfa5-21ca6f12711d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:06.467851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 186/373] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:06.473121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0669e-a7ae-4552-832e-5a36584aa259 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:06.508786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:06.557712 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d2e086-8e8c-4644-bfa5-21ca6f12711d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:06.560142 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d2e086-8e8c-4644-bfa5-21ca6f12711d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:06.641238 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0669e-a7ae-4552-832e-5a36584aa259 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:06.693930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:06.700779 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13022b-5349-438c-a9cf-40f88e596e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:06.701176 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 188/374] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:06.701950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13022b-5349-438c-a9cf-40f88e596e39 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:06.707343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13022b-5349-438c-a9cf-40f88e596e39 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:06.707343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13022b-5349-438c-a9cf-40f88e596e39 None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:06.707343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13022b-5349-438c-a9cf-40f88e596e39 None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:06.707617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2e086-8e8c-4644-bfa5-21ca6f12711d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:06.709799 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13022b-5349-438c-a9cf-40f88e596e39 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:06.714543 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 187/375] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:38:06 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:06.716977 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13022b-5349-438c-a9cf-40f88e596e39 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:06.855465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:06.893103 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13022b-5349-438c-a9cf-40f88e596e39 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:06.904952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2e086-8e8c-4644-bfa5-21ca6f12711d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:06.932090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb15ca-76f6-444a-a430-d8eb6a45c34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:06.932726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb15ca-76f6-444a-a430-d8eb6a45c34d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:06.933005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb15ca-76f6-444a-a430-d8eb6a45c34d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:06.933299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb15ca-76f6-444a-a430-d8eb6a45c34d None admin] PATH_INFO: `/v3/groups/fb4974a00d9c4982aee946511fde7fee` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:06.934402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb15ca-76f6-444a-a430-d8eb6a45c34d None admin] RBAC: Authorizing `identity:get_group(group_id=fb4974a00d9c4982aee946511fde7fee)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:06.937360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb15ca-76f6-444a-a430-d8eb6a45c34d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:06.939552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 188/376] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 30 14:38:06 2022] GET /identity/v3/groups/fb4974a00d9c4982aee946511fde7fee => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:06.944570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3f29f-7eda-428c-b83b-74512b9de6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:06.945009 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb15ca-76f6-444a-a430-d8eb6a45c34d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:06.945702 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3f29f-7eda-428c-b83b-74512b9de6bd None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:06.946177 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3f29f-7eda-428c-b83b-74512b9de6bd None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:06.946642 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3f29f-7eda-428c-b83b-74512b9de6bd None admin] PATH_INFO: `/v3/projects/d707a58a86f74ccbb5ffb499d9409139` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:06.949206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3f29f-7eda-428c-b83b-74512b9de6bd None admin] RBAC: Authorizing `identity:get_project(project_id=d707a58a86f74ccbb5ffb499d9409139)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:06.950922 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3f29f-7eda-428c-b83b-74512b9de6bd None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:06.953151 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 189/377] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:38:06 2022] GET /identity/v3/projects/d707a58a86f74ccbb5ffb499d9409139 => generated 344 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:06.958158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3f29f-7eda-428c-b83b-74512b9de6bd None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:07.060893 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb15ca-76f6-444a-a430-d8eb6a45c34d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:07.069361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:07.073647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3f29f-7eda-428c-b83b-74512b9de6bd None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:07.103145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50178578-1084-4a79-8cc9-7d72f1b9a386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:07.104918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50178578-1084-4a79-8cc9-7d72f1b9a386 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:07.105675 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50178578-1084-4a79-8cc9-7d72f1b9a386 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:07.106181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50178578-1084-4a79-8cc9-7d72f1b9a386 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:07.107926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50178578-1084-4a79-8cc9-7d72f1b9a386 None admin] RBAC: Adding query filter params (group.id=fb4974a00d9c4982aee946511fde7fee, role.id=6fd5089acea448219d4c0a9a9424a3af, scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:07.108492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50178578-1084-4a79-8cc9-7d72f1b9a386 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:07.110237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50178578-1084-4a79-8cc9-7d72f1b9a386 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:07.120612 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7d162-29fc-41d2-9d97-eecf494c4f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:07.121003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 189/378] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 30 14:38:06 2022] GET /identity/v3/role_assignments?group.id=fb4974a00d9c4982aee946511fde7fee&scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8&role.id=6fd5089acea448219d4c0a9a9424a3af => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:07.121229 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7d162-29fc-41d2-9d97-eecf494c4f22 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:07.121516 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7d162-29fc-41d2-9d97-eecf494c4f22 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:07.121793 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7d162-29fc-41d2-9d97-eecf494c4f22 None admin] PATH_INFO: `/v3/groups/73093109afa4403cb77794c287978bda` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:07.123171 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e99f1-ec58-4cd1-9f10-28a1012c9b76 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:07.123665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e99f1-ec58-4cd1-9f10-28a1012c9b76 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:07.124146 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e99f1-ec58-4cd1-9f10-28a1012c9b76 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:07.127296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 190/379] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:07.129910 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7d162-29fc-41d2-9d97-eecf494c4f22 None admin] RBAC: Authorizing `identity:get_group(group_id=73093109afa4403cb77794c287978bda)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:07.131322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7d162-29fc-41d2-9d97-eecf494c4f22 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:07.133219 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 190/380] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 30 14:38:06 2022] GET /identity/v3/groups/73093109afa4403cb77794c287978bda => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:07.134472 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851186f-6f18-45dc-b5bc-8139fc5cb0db None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:07.134757 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851186f-6f18-45dc-b5bc-8139fc5cb0db None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:07.135030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851186f-6f18-45dc-b5bc-8139fc5cb0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:07.144859 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7d162-29fc-41d2-9d97-eecf494c4f22 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:07.173736 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e851186f-6f18-45dc-b5bc-8139fc5cb0db None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:07.175035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e851186f-6f18-45dc-b5bc-8139fc5cb0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:07.203015 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:07.326216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 191/381] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:07.343846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7d162-29fc-41d2-9d97-eecf494c4f22 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:07.371614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:07.380088 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d689dd5-9d90-427f-8884-3f04fd52e7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:07.380730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689dd5-9d90-427f-8884-3f04fd52e7a8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:07.381013 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689dd5-9d90-427f-8884-3f04fd52e7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:07.381338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689dd5-9d90-427f-8884-3f04fd52e7a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:07.387087 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d689dd5-9d90-427f-8884-3f04fd52e7a8 None admin] RBAC: Adding query filter params (group.id=73093109afa4403cb77794c287978bda, role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=d707a58a86f74ccbb5ffb499d9409139) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:07.387087 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d689dd5-9d90-427f-8884-3f04fd52e7a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:07.387087 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d689dd5-9d90-427f-8884-3f04fd52e7a8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:07.389797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 191/382] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 30 14:38:07 2022] GET /identity/v3/role_assignments?group.id=73093109afa4403cb77794c287978bda&scope.project.id=d707a58a86f74ccbb5ffb499d9409139&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 260 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:07.618728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0ea56-c47f-4b55-a035-da6c5f40087e None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:07.619150 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0ea56-c47f-4b55-a035-da6c5f40087e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:07.619336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0ea56-c47f-4b55-a035-da6c5f40087e None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:07.620910 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 192/383] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:07.626917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0da5be-2f7c-4bce-a1cc-639d6ebf0301 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:07.627250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0da5be-2f7c-4bce-a1cc-639d6ebf0301 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:07.627530 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0da5be-2f7c-4bce-a1cc-639d6ebf0301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:07.640083 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0ea56-c47f-4b55-a035-da6c5f40087e None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:07.653824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6e0da5be-2f7c-4bce-a1cc-639d6ebf0301 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:07.654697 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6e0da5be-2f7c-4bce-a1cc-639d6ebf0301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:07.753109 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0ea56-c47f-4b55-a035-da6c5f40087e None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:07.790487 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ab2cb-5597-484b-b132-c7bbb1c96387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:07.791489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ab2cb-5597-484b-b132-c7bbb1c96387 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:07.791969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ab2cb-5597-484b-b132-c7bbb1c96387 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:07.792470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ab2cb-5597-484b-b132-c7bbb1c96387 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:07.803420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ab2cb-5597-484b-b132-c7bbb1c96387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:07.807292 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 192/384] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:07.809186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-af6ab2cb-5597-484b-b132-c7bbb1c96387 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:38:07.809186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:07.809186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:07.809186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:07.809186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:07.809186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:07.809186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:07.809186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:07.809186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:07.809186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:07.809186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:07.809186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:07.809186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:07.809186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:07.809186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:07.809186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:07.810560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:07.810560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:07.810560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:07.810560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:07.810560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:07.810560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:07.810560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:07.810560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:07.810560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:07.810560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:07.810560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:07.810560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:07.810560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:07.810560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:07.810560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:07.810560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:07.811377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:07.811377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:07.811377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:07.811377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:07.811377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:07.811377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:07.811377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:07.811377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:07.811377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:07.811377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:07.811377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:38:07.811377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:07.818392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 193/385] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:07.820279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0da5be-2f7c-4bce-a1cc-639d6ebf0301 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:07.944200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0da5be-2f7c-4bce-a1cc-639d6ebf0301 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:07.973131 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:07.991587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9acc93-1b81-4c13-9199-e7dc8ad9943a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:07.992201 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9acc93-1b81-4c13-9199-e7dc8ad9943a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:07.992480 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9acc93-1b81-4c13-9199-e7dc8ad9943a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:07.995327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9acc93-1b81-4c13-9199-e7dc8ad9943a None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:07.995327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9acc93-1b81-4c13-9199-e7dc8ad9943a None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:07.995327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9acc93-1b81-4c13-9199-e7dc8ad9943a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:07.995327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9acc93-1b81-4c13-9199-e7dc8ad9943a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:08.004990 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 193/386] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:38:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:08.009642 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ab2cb-5597-484b-b132-c7bbb1c96387 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:08.023488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:08.118135 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ab2cb-5597-484b-b132-c7bbb1c96387 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:08.154278 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc6873-b1bd-4051-ba2c-f9f217372a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:08.154908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc6873-b1bd-4051-ba2c-f9f217372a45 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:08.155208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc6873-b1bd-4051-ba2c-f9f217372a45 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:08.155479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc6873-b1bd-4051-ba2c-f9f217372a45 None admin] PATH_INFO: `/v3/groups/fb4974a00d9c4982aee946511fde7fee` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:08.156589 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc6873-b1bd-4051-ba2c-f9f217372a45 None admin] RBAC: Authorizing `identity:get_group(group_id=fb4974a00d9c4982aee946511fde7fee)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:08.158030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc6873-b1bd-4051-ba2c-f9f217372a45 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:08.162920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 194/387] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 30 14:38:08 2022] GET /identity/v3/groups/fb4974a00d9c4982aee946511fde7fee => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:08.165743 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9acc93-1b81-4c13-9199-e7dc8ad9943a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:08.269781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9acc93-1b81-4c13-9199-e7dc8ad9943a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:08.304858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae2624-2706-41e7-80cd-fd77916ac93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:08.305495 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae2624-2706-41e7-80cd-fd77916ac93d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:08.305776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae2624-2706-41e7-80cd-fd77916ac93d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:08.306052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae2624-2706-41e7-80cd-fd77916ac93d None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:08.307222 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae2624-2706-41e7-80cd-fd77916ac93d None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:08.308667 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae2624-2706-41e7-80cd-fd77916ac93d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:08.313987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 194/388] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:38:08 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:08.317002 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc6873-b1bd-4051-ba2c-f9f217372a45 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:08.398782 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:08.422825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc6873-b1bd-4051-ba2c-f9f217372a45 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:08.458847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae2624-2706-41e7-80cd-fd77916ac93d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:08.469308 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9929b3-44af-4615-975f-8a5bd518411b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:08.469985 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9929b3-44af-4615-975f-8a5bd518411b None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:08.470261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9929b3-44af-4615-975f-8a5bd518411b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:08.470534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9929b3-44af-4615-975f-8a5bd518411b None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4/groups/fb4974a00d9c4982aee946511fde7fee/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:08.476937 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9929b3-44af-4615-975f-8a5bd518411b None admin] RBAC: Authorizing `identity:create_grant(project_id=bc07ff985d9a458a94b1aefcf6147ae4, group_id=fb4974a00d9c4982aee946511fde7fee, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:08.478384 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9929b3-44af-4615-975f-8a5bd518411b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:08.525711 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:08.553154 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 195/389] 10.222.0.27 () {58 vars in 1442 bytes} [Tue Aug 30 14:38:08 2022] PUT /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4/groups/fb4974a00d9c4982aee946511fde7fee/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:38:08.596663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae2624-2706-41e7-80cd-fd77916ac93d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:08.643816 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e32b3-875a-4518-bbc1-8d8cf157e639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:08.644481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e32b3-875a-4518-bbc1-8d8cf157e639 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:08.644772 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e32b3-875a-4518-bbc1-8d8cf157e639 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:08.645042 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e32b3-875a-4518-bbc1-8d8cf157e639 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:08.658904 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382d0f8-a67e-42de-9582-af45cead1d64 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:08.659282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382d0f8-a67e-42de-9582-af45cead1d64 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:08.659534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382d0f8-a67e-42de-9582-af45cead1d64 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:08.661028 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 196/390] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:08.662581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e32b3-875a-4518-bbc1-8d8cf157e639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:08.666644 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7430c2-ff0a-4a23-a0c1-7861a114cddc None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:08.667184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7430c2-ff0a-4a23-a0c1-7861a114cddc None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:08.667674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7430c2-ff0a-4a23-a0c1-7861a114cddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:08.668322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-de2e32b3-875a-4518-bbc1-8d8cf157e639 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:38:08.668322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:08.668322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:08.668322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:08.668322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:08.668322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:08.668322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:08.668322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:08.668322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:08.668322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:08.668322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:08.668322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:08.668322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:08.668322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:08.668322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:08.668322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:08.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:08.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:08.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:08.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:08.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:08.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:08.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:08.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:08.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:08.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:08.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:08.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:08.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:08.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:08.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:08.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:08.670031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:08.670031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:08.670031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:08.670031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:08.670031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:08.670031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:08.670031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:08.670031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:08.670031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:08.670031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:08.670031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:38:08.670031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:08.673670 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 195/391] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:38:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:08.675895 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e32b3-875a-4518-bbc1-8d8cf157e639 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:08.695021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-9e7430c2-ff0a-4a23-a0c1-7861a114cddc None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:08.697438 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9e7430c2-ff0a-4a23-a0c1-7861a114cddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:08.795358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 197/392] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:38:08.876075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e32b3-875a-4518-bbc1-8d8cf157e639 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:08.898980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:08.932883 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96a5a5-e46d-44a9-983e-1daf0c46e841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:08.933719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96a5a5-e46d-44a9-983e-1daf0c46e841 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:08.934002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96a5a5-e46d-44a9-983e-1daf0c46e841 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:08.934270 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96a5a5-e46d-44a9-983e-1daf0c46e841 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:08.935724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96a5a5-e46d-44a9-983e-1daf0c46e841 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:08.937643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96a5a5-e46d-44a9-983e-1daf0c46e841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:08.939069 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96a5a5-e46d-44a9-983e-1daf0c46e841 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:08.948929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 196/393] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:08.954875 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7430c2-ff0a-4a23-a0c1-7861a114cddc None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:09.033669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:09.074521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7430c2-ff0a-4a23-a0c1-7861a114cddc None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:09.111390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f62e0-ddec-45ac-b008-6492d27697a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:09.112028 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f62e0-ddec-45ac-b008-6492d27697a4 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:09.112310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f62e0-ddec-45ac-b008-6492d27697a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:09.112585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f62e0-ddec-45ac-b008-6492d27697a4 None admin] PATH_INFO: `/v3/groups/fb4974a00d9c4982aee946511fde7fee` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:09.113704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f62e0-ddec-45ac-b008-6492d27697a4 None admin] RBAC: Authorizing `identity:get_group(group_id=fb4974a00d9c4982aee946511fde7fee)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:09.115187 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f62e0-ddec-45ac-b008-6492d27697a4 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:09.117081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 198/394] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 30 14:38:08 2022] GET /identity/v3/groups/fb4974a00d9c4982aee946511fde7fee => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:09.120071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96a5a5-e46d-44a9-983e-1daf0c46e841 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:09.123353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f62e0-ddec-45ac-b008-6492d27697a4 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:09.235663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96a5a5-e46d-44a9-983e-1daf0c46e841 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:09.250140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f62e0-ddec-45ac-b008-6492d27697a4 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:09.279202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042133b9-feae-4412-8dbf-8fa6522bfe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:09.280287 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042133b9-feae-4412-8dbf-8fa6522bfe3a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:09.280796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042133b9-feae-4412-8dbf-8fa6522bfe3a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:09.281268 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042133b9-feae-4412-8dbf-8fa6522bfe3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:09.294357 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042133b9-feae-4412-8dbf-8fa6522bfe3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:09.294976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09f00e-8f5c-4365-9b40-a7be8b22f65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:09.295620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09f00e-8f5c-4365-9b40-a7be8b22f65c None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:09.295902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09f00e-8f5c-4365-9b40-a7be8b22f65c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:09.296174 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09f00e-8f5c-4365-9b40-a7be8b22f65c None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:09.297354 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09f00e-8f5c-4365-9b40-a7be8b22f65c None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:09.298865 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09f00e-8f5c-4365-9b40-a7be8b22f65c None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:09.300557 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 199/395] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:38:09 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:09.302394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-042133b9-feae-4412-8dbf-8fa6522bfe3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:38:09.302394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:09.302394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:09.302394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:09.302394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:09.302394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:09.302394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:09.302394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:09.302394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:09.302394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:09.302394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:09.302394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:09.302394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:09.302394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:09.302394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:09.302394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:09.307754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:09.307754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:09.307754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:09.307754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:09.307754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:09.307754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:09.307754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:09.307754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:09.307754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:09.307754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:09.307754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:09.307754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:09.307754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:09.307754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:09.307754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:09.307754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:09.308546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:09.308546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:09.308546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:09.308546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:09.308546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:09.308546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:09.308546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:09.308546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:09.308546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:09.308546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:09.308546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:38:09.308546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:09.308546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 197/396] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:09.309215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09f00e-8f5c-4365-9b40-a7be8b22f65c None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:09.313161 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:09.314688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042133b9-feae-4412-8dbf-8fa6522bfe3a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:09.359667 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:09.421826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09f00e-8f5c-4365-9b40-a7be8b22f65c None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:09.425084 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042133b9-feae-4412-8dbf-8fa6522bfe3a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:09.458817 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa2e7a-0864-4838-88a6-3757cc6bf6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:09.459247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa2e7a-0864-4838-88a6-3757cc6bf6c5 None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:09.459533 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa2e7a-0864-4838-88a6-3757cc6bf6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:09.459811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa2e7a-0864-4838-88a6-3757cc6bf6c5 None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4/groups/fb4974a00d9c4982aee946511fde7fee/roles/6fd5089acea448219d4c0a9a9424a3af` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:09.462673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa2e7a-0864-4838-88a6-3757cc6bf6c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc07ff985d9a458a94b1aefcf6147ae4, group_id=fb4974a00d9c4982aee946511fde7fee, role_id=6fd5089acea448219d4c0a9a9424a3af)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:09.464200 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa2e7a-0864-4838-88a6-3757cc6bf6c5 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:09.469213 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66979817-900b-4e35-a3a3-dcfa4ba21106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:09.469910 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66979817-900b-4e35-a3a3-dcfa4ba21106 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:09.470264 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66979817-900b-4e35-a3a3-dcfa4ba21106 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:09.470611 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66979817-900b-4e35-a3a3-dcfa4ba21106 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:09.475140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66979817-900b-4e35-a3a3-dcfa4ba21106 None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:09.475140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66979817-900b-4e35-a3a3-dcfa4ba21106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:09.475140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66979817-900b-4e35-a3a3-dcfa4ba21106 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:09.482566 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 198/397] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:38:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:09.488174 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66979817-900b-4e35-a3a3-dcfa4ba21106 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:09.545607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 200/398] 10.222.0.27 () {58 vars in 1442 bytes} [Tue Aug 30 14:38:09 2022] PUT /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4/groups/fb4974a00d9c4982aee946511fde7fee/roles/6fd5089acea448219d4c0a9a9424a3af => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:38:09.595249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:09.602093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66979817-900b-4e35-a3a3-dcfa4ba21106 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:09.646345 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1168e7-e131-44b9-97d4-ab378f248aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:09.646912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1168e7-e131-44b9-97d4-ab378f248aeb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:09.647253 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1168e7-e131-44b9-97d4-ab378f248aeb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:09.647533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1168e7-e131-44b9-97d4-ab378f248aeb None admin] PATH_INFO: `/v3/groups/fb4974a00d9c4982aee946511fde7fee` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:09.648699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1168e7-e131-44b9-97d4-ab378f248aeb None admin] RBAC: Authorizing `identity:get_group(group_id=fb4974a00d9c4982aee946511fde7fee)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:09.650157 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1168e7-e131-44b9-97d4-ab378f248aeb None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:09.652241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 199/399] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 30 14:38:09 2022] GET /identity/v3/groups/fb4974a00d9c4982aee946511fde7fee => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:09.658604 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa2e7a-0864-4838-88a6-3757cc6bf6c5 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:09.754526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:09.795969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa2e7a-0864-4838-88a6-3757cc6bf6c5 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:09.824168 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6aa28c-2f34-4290-9414-e097b178ffec None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:09.824780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6aa28c-2f34-4290-9414-e097b178ffec None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:09.825836 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6aa28c-2f34-4290-9414-e097b178ffec None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:09.827392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 200/400] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:09.839174 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e8de1-305c-49ac-91ac-fc299fb3dac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:09.839174 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e8de1-305c-49ac-91ac-fc299fb3dac8 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:09.839418 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f9872-60bb-40c9-ad75-561b6689a6b6 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:09.839518 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e8de1-305c-49ac-91ac-fc299fb3dac8 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:09.839589 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f9872-60bb-40c9-ad75-561b6689a6b6 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:09.839654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e8de1-305c-49ac-91ac-fc299fb3dac8 None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:09.839718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f9872-60bb-40c9-ad75-561b6689a6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:09.839781 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e8de1-305c-49ac-91ac-fc299fb3dac8 None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:09.841323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e8de1-305c-49ac-91ac-fc299fb3dac8 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:09.844050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 201/401] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:38:09 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:09.848595 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e8de1-305c-49ac-91ac-fc299fb3dac8 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:09.872187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b83f9872-60bb-40c9-ad75-561b6689a6b6 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:09.873757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b83f9872-60bb-40c9-ad75-561b6689a6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:09.984795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 201/402] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:38:10.023404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e8de1-305c-49ac-91ac-fc299fb3dac8 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:10.057857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:10.064425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bd8f6-e275-496e-8a35-e842aa2aba78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:10.065094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bd8f6-e275-496e-8a35-e842aa2aba78 None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:10.065417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bd8f6-e275-496e-8a35-e842aa2aba78 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:10.065758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bd8f6-e275-496e-8a35-e842aa2aba78 None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8/groups/fb4974a00d9c4982aee946511fde7fee/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:10.067925 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bd8f6-e275-496e-8a35-e842aa2aba78 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8, group_id=fb4974a00d9c4982aee946511fde7fee, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:10.069427 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bd8f6-e275-496e-8a35-e842aa2aba78 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:10.116741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d4c6d-ec81-495e-b066-0c23549fc868 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:10.117049 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d4c6d-ec81-495e-b066-0c23549fc868 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:10.117407 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d4c6d-ec81-495e-b066-0c23549fc868 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:10.119027 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 202/403] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:10.125624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d28030-3544-4b0d-8629-cb7f5dcec39a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:10.125907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d28030-3544-4b0d-8629-cb7f5dcec39a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:10.126179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d28030-3544-4b0d-8629-cb7f5dcec39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:10.147155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c0d28030-3544-4b0d-8629-cb7f5dcec39a None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:10.148358 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c0d28030-3544-4b0d-8629-cb7f5dcec39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:10.172653 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 202/404] 10.222.0.27 () {58 vars in 1442 bytes} [Tue Aug 30 14:38:09 2022] PUT /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8/groups/fb4974a00d9c4982aee946511fde7fee/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:38:10.236658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 203/405] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:10.321480 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bd8f6-e275-496e-8a35-e842aa2aba78 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:10.459315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bd8f6-e275-496e-8a35-e842aa2aba78 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:10.502279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4fbd7b-db42-450d-8a8f-4d503dbbd0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:10.502837 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fbd7b-db42-450d-8a8f-4d503dbbd0bb None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:10.503133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fbd7b-db42-450d-8a8f-4d503dbbd0bb None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:10.503409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fbd7b-db42-450d-8a8f-4d503dbbd0bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:10.515947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4fbd7b-db42-450d-8a8f-4d503dbbd0bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:10.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-ed4fbd7b-db42-450d-8a8f-4d503dbbd0bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:38:10.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:10.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:10.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:10.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:10.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:10.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:10.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:10.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:10.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:10.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:10.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:10.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:10.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:10.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:10.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:10.522147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:10.522147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:10.522147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:10.522147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:10.522147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:10.522147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:10.522147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:10.522147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:10.522147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:10.522147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:10.522147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:10.522147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:10.522147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:10.522147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:10.522147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:10.522147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:10.522942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:10.522942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:10.522942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:10.522942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:10.522942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:10.522942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:10.522942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:10.522942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:10.522942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:10.522942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:10.522942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:38:10.522942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:10.522942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 203/406] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:38:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:10.528762 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d28030-3544-4b0d-8629-cb7f5dcec39a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:10.574478 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:10.593603 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4fbd7b-db42-450d-8a8f-4d503dbbd0bb None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:10.695463 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:10.729056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d28030-3544-4b0d-8629-cb7f5dcec39a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:10.770093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4fbd7b-db42-450d-8a8f-4d503dbbd0bb None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:10.781808 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765f340-a417-4fd4-8704-5891703e3430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:10.782419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765f340-a417-4fd4-8704-5891703e3430 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:10.782699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765f340-a417-4fd4-8704-5891703e3430 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:10.782967 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765f340-a417-4fd4-8704-5891703e3430 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:10.783671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765f340-a417-4fd4-8704-5891703e3430 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:10.783969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765f340-a417-4fd4-8704-5891703e3430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:10.785360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765f340-a417-4fd4-8704-5891703e3430 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:10.798872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 204/407] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:10.810110 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765f340-a417-4fd4-8704-5891703e3430 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:10.812007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced81f3-6dee-45b6-9bc3-5d450b68bc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:10.812640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced81f3-6dee-45b6-9bc3-5d450b68bc7a None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:10.812914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced81f3-6dee-45b6-9bc3-5d450b68bc7a None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:10.813184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced81f3-6dee-45b6-9bc3-5d450b68bc7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:10.824626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced81f3-6dee-45b6-9bc3-5d450b68bc7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:10.830039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-8ced81f3-6dee-45b6-9bc3-5d450b68bc7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:10.830039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:10.830039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:10.830039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:10.830039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:10.830039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:10.830039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:10.830039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:10.830039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:10.830039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:10.830039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:10.830039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:10.830039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:10.830039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:10.830039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:10.830039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:10.830901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:10.830901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:10.830901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:10.830901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:10.830901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:10.830901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:10.830901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:10.830901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:10.830901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:10.830901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:10.830901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:10.830901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:10.830901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:10.830901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:10.830901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:10.830901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:10.831722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:10.831722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:10.831722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:10.831722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:10.831722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:10.831722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:10.831722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:10.831722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:10.831722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:10.831722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:10.831722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:10.831722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:10.831722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 204/408] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:38:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:38:10.841801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced81f3-6dee-45b6-9bc3-5d450b68bc7a None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:10.928465 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:10.946696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765f340-a417-4fd4-8704-5891703e3430 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:10.976066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced81f3-6dee-45b6-9bc3-5d450b68bc7a None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:10.994008 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:11.014539 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4003f8-0872-4c34-942d-26b4c36c6436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:11.015288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4003f8-0872-4c34-942d-26b4c36c6436 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:11.015576 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4003f8-0872-4c34-942d-26b4c36c6436 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:11.015847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4003f8-0872-4c34-942d-26b4c36c6436 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:11.016520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4003f8-0872-4c34-942d-26b4c36c6436 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:11.016827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4003f8-0872-4c34-942d-26b4c36c6436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:11.018211 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4003f8-0872-4c34-942d-26b4c36c6436 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:11.022196 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99171152-82a5-42f8-8082-08cf1d16857b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:11.022792 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99171152-82a5-42f8-8082-08cf1d16857b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:11.023076 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99171152-82a5-42f8-8082-08cf1d16857b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:11.023375 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99171152-82a5-42f8-8082-08cf1d16857b None admin] PATH_INFO: `/v3/groups/fb4974a00d9c4982aee946511fde7fee` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:11.024520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99171152-82a5-42f8-8082-08cf1d16857b None admin] RBAC: Authorizing `identity:get_group(group_id=fb4974a00d9c4982aee946511fde7fee)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:11.025965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99171152-82a5-42f8-8082-08cf1d16857b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:11.028697 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 205/409] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 30 14:38:10 2022] GET /identity/v3/groups/fb4974a00d9c4982aee946511fde7fee => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:11.035285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99171152-82a5-42f8-8082-08cf1d16857b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:11.036550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 205/410] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:11.041756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4003f8-0872-4c34-942d-26b4c36c6436 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:11.177502 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4003f8-0872-4c34-942d-26b4c36c6436 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:11.214530 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062df6a3-204f-4ef0-825f-938a968b6c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:11.215187 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062df6a3-204f-4ef0-825f-938a968b6c8e None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:11.215519 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062df6a3-204f-4ef0-825f-938a968b6c8e None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:11.215797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062df6a3-204f-4ef0-825f-938a968b6c8e None admin] PATH_INFO: `/v3/groups/73093109afa4403cb77794c287978bda` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:11.216924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062df6a3-204f-4ef0-825f-938a968b6c8e None admin] RBAC: Authorizing `identity:get_group(group_id=73093109afa4403cb77794c287978bda)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:11.218484 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062df6a3-204f-4ef0-825f-938a968b6c8e None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:11.220445 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 206/411] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 30 14:38:11 2022] GET /identity/v3/groups/73093109afa4403cb77794c287978bda => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:11.229734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062df6a3-204f-4ef0-825f-938a968b6c8e None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:11.288433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99171152-82a5-42f8-8082-08cf1d16857b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:11.319435 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:11.334550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07107104-3915-4846-ad9a-3eed004def89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:11.337614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07107104-3915-4846-ad9a-3eed004def89 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:11.337896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07107104-3915-4846-ad9a-3eed004def89 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:11.338174 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07107104-3915-4846-ad9a-3eed004def89 None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:11.339376 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07107104-3915-4846-ad9a-3eed004def89 None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:11.340913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07107104-3915-4846-ad9a-3eed004def89 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:11.344057 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 206/412] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:38:11 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:11.352080 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07107104-3915-4846-ad9a-3eed004def89 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:11.394129 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:11.425713 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062df6a3-204f-4ef0-825f-938a968b6c8e None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:11.460913 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64827bc8-7770-4219-b12f-e42d002be52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:11.461641 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64827bc8-7770-4219-b12f-e42d002be52f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:11.462088 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64827bc8-7770-4219-b12f-e42d002be52f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:11.462449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64827bc8-7770-4219-b12f-e42d002be52f None admin] PATH_INFO: `/v3/projects/d707a58a86f74ccbb5ffb499d9409139` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:11.463673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64827bc8-7770-4219-b12f-e42d002be52f None admin] RBAC: Authorizing `identity:get_project(project_id=d707a58a86f74ccbb5ffb499d9409139)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:11.465289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64827bc8-7770-4219-b12f-e42d002be52f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:11.467980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 207/413] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:38:11 2022] GET /identity/v3/projects/d707a58a86f74ccbb5ffb499d9409139 => generated 344 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:11.469295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2317e-bdb4-4897-89e3-8a614c639082 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:11.469637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2317e-bdb4-4897-89e3-8a614c639082 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:11.470073 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2317e-bdb4-4897-89e3-8a614c639082 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:11.471501 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 208/414] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:11.475978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2317e-bdb4-4897-89e3-8a614c639082 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:11.477125 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07107104-3915-4846-ad9a-3eed004def89 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:11.520407 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f3baa-5f14-4fa8-87f6-a399129c9691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:11.521354 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f3baa-5f14-4fa8-87f6-a399129c9691 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:11.522278 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f3baa-5f14-4fa8-87f6-a399129c9691 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:11.522732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f3baa-5f14-4fa8-87f6-a399129c9691 None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8/groups/fb4974a00d9c4982aee946511fde7fee/roles/6fd5089acea448219d4c0a9a9424a3af` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:11.526064 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f3baa-5f14-4fa8-87f6-a399129c9691 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8, group_id=fb4974a00d9c4982aee946511fde7fee, role_id=6fd5089acea448219d4c0a9a9424a3af)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:11.528849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f3baa-5f14-4fa8-87f6-a399129c9691 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:11.585234 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2317e-bdb4-4897-89e3-8a614c639082 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:11.595767 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:11.613457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 207/415] 10.222.0.27 () {58 vars in 1442 bytes} [Tue Aug 30 14:38:11 2022] PUT /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8/groups/fb4974a00d9c4982aee946511fde7fee/roles/6fd5089acea448219d4c0a9a9424a3af => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:38:11.617577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f397698-7226-4266-9cdd-3d2e7cb86b81 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:11.617857 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f397698-7226-4266-9cdd-3d2e7cb86b81 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:11.618353 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f397698-7226-4266-9cdd-3d2e7cb86b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:11.651317 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d6082-6a7b-4dba-b93c-060cc9d1f4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:11.652290 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d6082-6a7b-4dba-b93c-060cc9d1f4f0 None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:11.652564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d6082-6a7b-4dba-b93c-060cc9d1f4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:11.652831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d6082-6a7b-4dba-b93c-060cc9d1f4f0 None admin] PATH_INFO: `/v3/projects/d707a58a86f74ccbb5ffb499d9409139/groups/73093109afa4403cb77794c287978bda/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:11.658787 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d6082-6a7b-4dba-b93c-060cc9d1f4f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d707a58a86f74ccbb5ffb499d9409139, group_id=73093109afa4403cb77794c287978bda, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:11.661153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5f397698-7226-4266-9cdd-3d2e7cb86b81 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:11.662380 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5f397698-7226-4266-9cdd-3d2e7cb86b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:11.663678 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d6082-6a7b-4dba-b93c-060cc9d1f4f0 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:11.752732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 208/416] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:11.768722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 209/417] 10.222.0.27 () {58 vars in 1442 bytes} [Tue Aug 30 14:38:11 2022] PUT /identity/v3/projects/d707a58a86f74ccbb5ffb499d9409139/groups/73093109afa4403cb77794c287978bda/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:38:12.039145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f397698-7226-4266-9cdd-3d2e7cb86b81 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:12.054176 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:12.110050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b8886-6de6-4938-ade9-80ea0e576b3c None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:12.110370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b8886-6de6-4938-ade9-80ea0e576b3c None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:12.110608 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b8886-6de6-4938-ade9-80ea0e576b3c None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:12.112283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 210/418] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:38:12.117352 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667f902-6627-46c9-8ecb-385b5fcb9801 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:12.117573 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667f902-6627-46c9-8ecb-385b5fcb9801 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:12.117846 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667f902-6627-46c9-8ecb-385b5fcb9801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:12.145003 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f397698-7226-4266-9cdd-3d2e7cb86b81 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:12.164096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-2667f902-6627-46c9-8ecb-385b5fcb9801 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:12.165312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-2667f902-6627-46c9-8ecb-385b5fcb9801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:12.191609 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05ba23-5543-47ff-ab56-41bbba9db09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:12.192242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05ba23-5543-47ff-ab56-41bbba9db09b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:12.192514 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05ba23-5543-47ff-ab56-41bbba9db09b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:12.192778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05ba23-5543-47ff-ab56-41bbba9db09b None admin] PATH_INFO: `/v3/roles/member` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:12.214175 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e05ba23-5543-47ff-ab56-41bbba9db09b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:12.218980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-9e05ba23-5543-47ff-ab56-41bbba9db09b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:38:12.218980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:12.218980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:12.218980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:12.218980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:12.218980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:12.218980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:12.218980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:12.218980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:12.218980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:12.218980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:12.218980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:12.218980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:12.218980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:12.218980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:12.218980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:12.219842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:12.219842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:12.219842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:12.219842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:12.219842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:12.219842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:12.219842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:12.219842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:12.219842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:12.219842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:12.219842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:12.219842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:12.219842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:12.219842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:12.219842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:12.219842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:12.220646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:12.220646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:12.220646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:12.220646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:12.220646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:12.220646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:12.220646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:12.220646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:12.220646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:12.220646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:12.220646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:38:12.220646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:12.221777 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 209/419] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:12.230330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05ba23-5543-47ff-ab56-41bbba9db09b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:12.305960 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 211/420] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:38:12.318051 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:12.350167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05ba23-5543-47ff-ab56-41bbba9db09b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:12.389311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe09ce-c537-4bd9-9245-d15d20fd93df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:12.389918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe09ce-c537-4bd9-9245-d15d20fd93df None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:12.390407 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe09ce-c537-4bd9-9245-d15d20fd93df None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:12.390677 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe09ce-c537-4bd9-9245-d15d20fd93df None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:12.391372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe09ce-c537-4bd9-9245-d15d20fd93df None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:12.391671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe09ce-c537-4bd9-9245-d15d20fd93df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:12.393061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe09ce-c537-4bd9-9245-d15d20fd93df None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:12.402318 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 210/421] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:38:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:12.407921 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667f902-6627-46c9-8ecb-385b5fcb9801 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:12.504761 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08bcda6-1057-4769-9f20-daaa43c64132 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:12.505099 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08bcda6-1057-4769-9f20-daaa43c64132 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:12.505310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08bcda6-1057-4769-9f20-daaa43c64132 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:12.506760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 211/422] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:38:12.512054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe9f14-3ea3-4c21-a0ac-ba9ddc3576f2 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:12.513205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe9f14-3ea3-4c21-a0ac-ba9ddc3576f2 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:12.513694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe9f14-3ea3-4c21-a0ac-ba9ddc3576f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:12.533123 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bfbe9f14-3ea3-4c21-a0ac-ba9ddc3576f2 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:12.534614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bfbe9f14-3ea3-4c21-a0ac-ba9ddc3576f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:12.543222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:12.548116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667f902-6627-46c9-8ecb-385b5fcb9801 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:12.611284 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6ef0a-6691-4205-953d-f774239a99b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:12.613037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6ef0a-6691-4205-953d-f774239a99b7 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:12.613691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6ef0a-6691-4205-953d-f774239a99b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:12.615310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6ef0a-6691-4205-953d-f774239a99b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:12.647126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 212/423] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:12.647126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe9f14-3ea3-4c21-a0ac-ba9ddc3576f2 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:12.734940 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:12.756263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6ef0a-6691-4205-953d-f774239a99b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:12.758855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6ef0a-6691-4205-953d-f774239a99b7 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:12.764672 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe9f14-3ea3-4c21-a0ac-ba9ddc3576f2 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:12.834141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f28957-07e7-4d67-a64b-8e0459c66c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:12.835168 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f28957-07e7-4d67-a64b-8e0459c66c87 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:12.835660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f28957-07e7-4d67-a64b-8e0459c66c87 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:12.836134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f28957-07e7-4d67-a64b-8e0459c66c87 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:12.872627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f28957-07e7-4d67-a64b-8e0459c66c87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:12.881433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-47f28957-07e7-4d67-a64b-8e0459c66c87 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:38:12.881433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:12.881433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:12.881433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:12.881433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:12.881433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:12.881433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:12.881433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:12.881433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:12.881433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:12.881433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:12.881433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:12.881433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:12.881433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:12.881433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:12.881433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:12.882420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:12.882420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:12.882420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:12.882420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:12.882420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:12.882420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:12.882420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:12.882420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:12.882420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:12.882420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:12.882420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:12.882420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:12.882420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:12.882420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:12.882420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:12.882420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:12.883293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:12.883293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:12.883293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:12.883293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:12.883293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:12.883293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:12.883293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:12.883293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:12.883293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:12.883293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:12.883293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:38:12.883293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:12.883293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 213/424] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:38:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:12.895069 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:12.900740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 212/425] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:12.901481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f28957-07e7-4d67-a64b-8e0459c66c87 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:12.916087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6ef0a-6691-4205-953d-f774239a99b7 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:13.012715 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f28957-07e7-4d67-a64b-8e0459c66c87 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:13.038251 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:13.048784 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6ef0a-6691-4205-953d-f774239a99b7 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:13.052966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2698eef-60c1-48d7-95e1-ee8b4eb8cdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:13.053645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2698eef-60c1-48d7-95e1-ee8b4eb8cdb4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:13.053985 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2698eef-60c1-48d7-95e1-ee8b4eb8cdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:13.054342 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2698eef-60c1-48d7-95e1-ee8b4eb8cdb4 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:13.055311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2698eef-60c1-48d7-95e1-ee8b4eb8cdb4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:13.061601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2698eef-60c1-48d7-95e1-ee8b4eb8cdb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:13.061601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2698eef-60c1-48d7-95e1-ee8b4eb8cdb4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:13.066730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 214/426] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:13.067749 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2698eef-60c1-48d7-95e1-ee8b4eb8cdb4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:13.085844 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ef6e6-3861-4ea6-85f2-e1af2c26627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:13.086477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ef6e6-3861-4ea6-85f2-e1af2c26627f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:13.086752 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ef6e6-3861-4ea6-85f2-e1af2c26627f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:13.087142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ef6e6-3861-4ea6-85f2-e1af2c26627f None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:13.088461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ef6e6-3861-4ea6-85f2-e1af2c26627f None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:13.089913 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ef6e6-3861-4ea6-85f2-e1af2c26627f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:13.092472 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 213/427] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:38:12 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:13.093235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ef6e6-3861-4ea6-85f2-e1af2c26627f None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:13.162162 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:13.183405 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2698eef-60c1-48d7-95e1-ee8b4eb8cdb4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:13.199977 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ef6e6-3861-4ea6-85f2-e1af2c26627f None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:13.221569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ac633-4341-4f34-8c29-4e90cf0b4230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:13.222252 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ac633-4341-4f34-8c29-4e90cf0b4230 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:13.222531 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ac633-4341-4f34-8c29-4e90cf0b4230 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:13.222796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ac633-4341-4f34-8c29-4e90cf0b4230 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:13.234442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ac633-4341-4f34-8c29-4e90cf0b4230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:13.239711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c013db-fda7-459d-8ab6-1749e458e397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:13.239860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-555ac633-4341-4f34-8c29-4e90cf0b4230 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:38:13.239860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:13.239860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:13.239860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:13.239860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:13.239860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:13.239860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:13.239860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:13.239860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:13.239860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:13.239860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:13.239860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:13.239860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:13.239860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:13.239860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:13.239860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:13.240694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:13.240694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:13.240694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:13.240694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:13.240694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:13.240694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:13.240694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:13.240694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:13.240694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:13.240694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:13.240694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:13.240694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:13.240694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:13.240694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:13.240694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:13.240694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:13.241489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:13.241489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:13.241489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:13.241489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:13.241489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:13.241489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:13.241489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:13.241489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:13.241489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:13.241489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:13.241489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:38:13.241489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:13.243254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c013db-fda7-459d-8ab6-1749e458e397 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:13.243745 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c013db-fda7-459d-8ab6-1749e458e397 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:13.244223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c013db-fda7-459d-8ab6-1749e458e397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:13.245675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 215/428] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:13.246487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ac633-4341-4f34-8c29-4e90cf0b4230 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:13.301243 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:13.323889 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c013db-fda7-459d-8ab6-1749e458e397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:13.326492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c013db-fda7-459d-8ab6-1749e458e397 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:13.359882 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ac633-4341-4f34-8c29-4e90cf0b4230 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:13.396248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:13.396452 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d842f-4905-4904-94b4-d8332df6560a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:13.398315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d842f-4905-4904-94b4-d8332df6560a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:13.398315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d842f-4905-4904-94b4-d8332df6560a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:13.398625 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d842f-4905-4904-94b4-d8332df6560a None admin] PATH_INFO: `/v3/groups/fb4974a00d9c4982aee946511fde7fee` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:13.399970 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d842f-4905-4904-94b4-d8332df6560a None admin] RBAC: Authorizing `identity:get_group(group_id=fb4974a00d9c4982aee946511fde7fee)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:13.401824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d842f-4905-4904-94b4-d8332df6560a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:13.409350 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 216/429] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 30 14:38:13 2022] GET /identity/v3/groups/fb4974a00d9c4982aee946511fde7fee => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:13.411866 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d842f-4905-4904-94b4-d8332df6560a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:13.436598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 214/430] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:13.437793 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c013db-fda7-459d-8ab6-1749e458e397 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:13.520963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d842f-4905-4904-94b4-d8332df6560a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:13.537734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:13.545901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c013db-fda7-459d-8ab6-1749e458e397 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:13.564584 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08384e-c241-46a5-bfa5-b146d888216f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:13.570938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08384e-c241-46a5-bfa5-b146d888216f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:13.570938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08384e-c241-46a5-bfa5-b146d888216f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:13.570938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08384e-c241-46a5-bfa5-b146d888216f None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:13.570938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08384e-c241-46a5-bfa5-b146d888216f None admin] RBAC: Adding query filter params (name=member) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:13.570938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08384e-c241-46a5-bfa5-b146d888216f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:13.570938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08384e-c241-46a5-bfa5-b146d888216f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:13.577807 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 217/431] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:38:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:13.579036 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08384e-c241-46a5-bfa5-b146d888216f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:13.583665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc98f84-deae-40cc-a316-3dcf6220c9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:13.584574 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc98f84-deae-40cc-a316-3dcf6220c9a7 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:13.585058 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc98f84-deae-40cc-a316-3dcf6220c9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:13.585585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc98f84-deae-40cc-a316-3dcf6220c9a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:13.586513 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc98f84-deae-40cc-a316-3dcf6220c9a7 None admin] RBAC: Adding query filter params (group.id=fb4974a00d9c4982aee946511fde7fee, role.id=7c4463788f534454b6d8319dbc4d2afc, scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:13.587020 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc98f84-deae-40cc-a316-3dcf6220c9a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:13.588764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc98f84-deae-40cc-a316-3dcf6220c9a7 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:13.595933 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 215/432] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 30 14:38:13 2022] GET /identity/v3/role_assignments?group.id=fb4974a00d9c4982aee946511fde7fee&scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4&role.id=7c4463788f534454b6d8319dbc4d2afc => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:13.596951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc98f84-deae-40cc-a316-3dcf6220c9a7 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:13.643126 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:13.694236 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08384e-c241-46a5-bfa5-b146d888216f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:13.772905 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696798c-a313-40a0-af7a-7faf32cea873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:13.773548 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696798c-a313-40a0-af7a-7faf32cea873 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:13.773821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696798c-a313-40a0-af7a-7faf32cea873 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:13.774085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696798c-a313-40a0-af7a-7faf32cea873 None admin] PATH_INFO: `/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:13.776110 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696798c-a313-40a0-af7a-7faf32cea873 None admin] RBAC: Authorizing `identity:get_project(project_id=bc07ff985d9a458a94b1aefcf6147ae4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:13.777878 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696798c-a313-40a0-af7a-7faf32cea873 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:13.783983 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 218/433] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:38:13 2022] GET /identity/v3/projects/bc07ff985d9a458a94b1aefcf6147ae4 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:13.785823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc98f84-deae-40cc-a316-3dcf6220c9a7 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:13.788416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696798c-a313-40a0-af7a-7faf32cea873 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:13.849177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4113a4-df7d-4c89-bb4e-a035d2feacd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:13.849782 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4113a4-df7d-4c89-bb4e-a035d2feacd9 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:13.850065 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4113a4-df7d-4c89-bb4e-a035d2feacd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:13.850576 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4113a4-df7d-4c89-bb4e-a035d2feacd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:13.866893 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:13.948337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696798c-a313-40a0-af7a-7faf32cea873 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:13.949296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4113a4-df7d-4c89-bb4e-a035d2feacd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:13.954936 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4113a4-df7d-4c89-bb4e-a035d2feacd9 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:13.990062 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:14.003950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed92458-307b-4ba4-a91b-61e87ca4a46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:14.004580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed92458-307b-4ba4-a91b-61e87ca4a46c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:14.005547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed92458-307b-4ba4-a91b-61e87ca4a46c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:14.005547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed92458-307b-4ba4-a91b-61e87ca4a46c None admin] PATH_INFO: `/v3/groups/fb4974a00d9c4982aee946511fde7fee` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:14.006676 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed92458-307b-4ba4-a91b-61e87ca4a46c None admin] RBAC: Authorizing `identity:get_group(group_id=fb4974a00d9c4982aee946511fde7fee)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:14.007896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed92458-307b-4ba4-a91b-61e87ca4a46c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:14.009815 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 219/434] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 30 14:38:13 2022] GET /identity/v3/groups/fb4974a00d9c4982aee946511fde7fee => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:14.010981 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2598c43-e00e-49ab-a746-a6afba4de87b None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:14.011295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2598c43-e00e-49ab-a746-a6afba4de87b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:14.011555 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2598c43-e00e-49ab-a746-a6afba4de87b None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:14.013609 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 220/435] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:14.017834 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2598c43-e00e-49ab-a746-a6afba4de87b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:14.082769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 216/436] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:14.084810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e585a2a-3186-4237-af11-1b05a16c4df2 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:14.085669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e585a2a-3186-4237-af11-1b05a16c4df2 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:14.086294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e585a2a-3186-4237-af11-1b05a16c4df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:14.107401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-0e585a2a-3186-4237-af11-1b05a16c4df2 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:14.108857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0e585a2a-3186-4237-af11-1b05a16c4df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:14.146516 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2598c43-e00e-49ab-a746-a6afba4de87b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:14.164665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:14.182832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 217/437] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:14.183779 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e585a2a-3186-4237-af11-1b05a16c4df2 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:14.189860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156870b-5725-4971-a1e6-b3f40021aa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:14.190466 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156870b-5725-4971-a1e6-b3f40021aa93 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:14.190743 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156870b-5725-4971-a1e6-b3f40021aa93 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:14.191017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156870b-5725-4971-a1e6-b3f40021aa93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:14.191834 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156870b-5725-4971-a1e6-b3f40021aa93 None admin] RBAC: Adding query filter params (group.id=fb4974a00d9c4982aee946511fde7fee, role.id=6fd5089acea448219d4c0a9a9424a3af, scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:14.192913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156870b-5725-4971-a1e6-b3f40021aa93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:14.195457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156870b-5725-4971-a1e6-b3f40021aa93 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:14.211623 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 221/438] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 30 14:38:14 2022] GET /identity/v3/role_assignments?group.id=fb4974a00d9c4982aee946511fde7fee&scope.project.id=bc07ff985d9a458a94b1aefcf6147ae4&role.id=6fd5089acea448219d4c0a9a9424a3af => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:14.213849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d3324-c3fd-4b38-861b-513b19e2041b None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:14.214132 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d3324-c3fd-4b38-861b-513b19e2041b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:14.214402 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d3324-c3fd-4b38-861b-513b19e2041b None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:14.216938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 222/439] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:14.225241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f94dc0-93a2-4b6e-ad26-a37f7b3cef6e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:14.225241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f94dc0-93a2-4b6e-ad26-a37f7b3cef6e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:14.225241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f94dc0-93a2-4b6e-ad26-a37f7b3cef6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:14.255974 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-d9f94dc0-93a2-4b6e-ad26-a37f7b3cef6e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:14.257479 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d9f94dc0-93a2-4b6e-ad26-a37f7b3cef6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:14.258742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:14.319915 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e585a2a-3186-4237-af11-1b05a16c4df2 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:14.340192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 223/440] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:14.363698 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab1315-25eb-4c16-82f7-5ee1a507ac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:14.364902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab1315-25eb-4c16-82f7-5ee1a507ac9d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:14.365320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab1315-25eb-4c16-82f7-5ee1a507ac9d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:14.365727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab1315-25eb-4c16-82f7-5ee1a507ac9d None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:14.366972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab1315-25eb-4c16-82f7-5ee1a507ac9d None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:14.368590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab1315-25eb-4c16-82f7-5ee1a507ac9d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:14.370704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 218/441] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:38:14 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:14.376144 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f94dc0-93a2-4b6e-ad26-a37f7b3cef6e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:14.465549 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab1315-25eb-4c16-82f7-5ee1a507ac9d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:14.482854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f94dc0-93a2-4b6e-ad26-a37f7b3cef6e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:14.493193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:14.525876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54aed8-c813-40a2-a843-373f381f04e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:14.527249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54aed8-c813-40a2-a843-373f381f04e9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:14.527734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54aed8-c813-40a2-a843-373f381f04e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:14.528314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54aed8-c813-40a2-a843-373f381f04e9 None admin] PATH_INFO: `/v3/groups/fb4974a00d9c4982aee946511fde7fee` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:14.529638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54aed8-c813-40a2-a843-373f381f04e9 None admin] RBAC: Authorizing `identity:get_group(group_id=fb4974a00d9c4982aee946511fde7fee)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:14.531464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54aed8-c813-40a2-a843-373f381f04e9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:14.533246 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 224/442] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 30 14:38:14 2022] GET /identity/v3/groups/fb4974a00d9c4982aee946511fde7fee => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:14.543684 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54aed8-c813-40a2-a843-373f381f04e9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:14.590045 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab1315-25eb-4c16-82f7-5ee1a507ac9d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:14.624505 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:14.636450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e703377-9397-464a-9a0c-7c2572816a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:14.637149 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e703377-9397-464a-9a0c-7c2572816a11 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:14.637979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e703377-9397-464a-9a0c-7c2572816a11 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:14.638254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e703377-9397-464a-9a0c-7c2572816a11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:14.649524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e703377-9397-464a-9a0c-7c2572816a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:14.654833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-7e703377-9397-464a-9a0c-7c2572816a11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:38:14.654833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:14.654833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:14.654833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:14.654833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:14.654833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:14.654833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:14.654833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:14.654833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:14.654833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:14.654833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:14.654833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:14.654833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:14.654833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:14.654833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:14.654833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:14.655914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:14.655914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:14.655914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:14.655914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:14.655914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:14.655914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:14.655914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:14.655914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:14.655914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:14.655914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:14.655914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:14.655914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:14.655914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:14.655914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:14.655914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:14.655914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:14.656833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:14.656833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:14.656833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:14.656833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:14.656833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:14.656833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:14.656833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:14.656833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:14.656833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:14.656833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:14.656833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:38:14.656833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:14.659857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 219/443] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:38:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:38:14.660799 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e703377-9397-464a-9a0c-7c2572816a11 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:14.666034 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54aed8-c813-40a2-a843-373f381f04e9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:14.706305 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1f86a-13fe-4ef8-b2cf-e5d1b849c9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:14.706923 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1f86a-13fe-4ef8-b2cf-e5d1b849c9ed None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:14.707235 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1f86a-13fe-4ef8-b2cf-e5d1b849c9ed None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:14.707504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1f86a-13fe-4ef8-b2cf-e5d1b849c9ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:14.708255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1f86a-13fe-4ef8-b2cf-e5d1b849c9ed None admin] RBAC: Adding query filter params (group.id=fb4974a00d9c4982aee946511fde7fee, role.id=7c4463788f534454b6d8319dbc4d2afc, scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:14.708566 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1f86a-13fe-4ef8-b2cf-e5d1b849c9ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:14.710023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1f86a-13fe-4ef8-b2cf-e5d1b849c9ed None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:14.716149 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 225/444] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 30 14:38:14 2022] GET /identity/v3/role_assignments?group.id=fb4974a00d9c4982aee946511fde7fee&scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8&role.id=7c4463788f534454b6d8319dbc4d2afc => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:14.716880 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1f86a-13fe-4ef8-b2cf-e5d1b849c9ed None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:14.761299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:14.780162 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e703377-9397-464a-9a0c-7c2572816a11 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:14.821964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733330e2-ceac-484f-8ecb-3b5a7ce6768e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:14.822642 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733330e2-ceac-484f-8ecb-3b5a7ce6768e None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:14.822918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733330e2-ceac-484f-8ecb-3b5a7ce6768e None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:14.823216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733330e2-ceac-484f-8ecb-3b5a7ce6768e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:14.831826 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1f86a-13fe-4ef8-b2cf-e5d1b849c9ed None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:14.835060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733330e2-ceac-484f-8ecb-3b5a7ce6768e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:14.840534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-733330e2-ceac-484f-8ecb-3b5a7ce6768e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:14.840534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:14.840534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:14.840534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:14.840534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:14.840534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:14.840534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:14.840534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:14.840534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:14.840534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:14.840534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:14.840534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:14.840534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:14.840534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:14.840534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:14.840534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:14.841452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:14.841452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:14.841452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:14.841452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:14.841452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:14.841452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:14.841452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:14.841452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:14.841452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:14.841452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:14.841452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:14.841452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:14.841452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:14.841452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:14.841452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:14.841452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:14.843302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:14.843302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:14.843302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:14.843302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:14.843302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:14.843302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:14.843302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:14.843302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:14.843302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:14.843302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:14.843302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:14.843302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:14.843302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 220/445] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:38:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:14.847829 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733330e2-ceac-484f-8ecb-3b5a7ce6768e None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:14.870921 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac253a78-e32f-4353-ba88-4bab25c6fc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:14.871636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac253a78-e32f-4353-ba88-4bab25c6fc39 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:14.872424 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac253a78-e32f-4353-ba88-4bab25c6fc39 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:14.872701 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac253a78-e32f-4353-ba88-4bab25c6fc39 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:14.873379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac253a78-e32f-4353-ba88-4bab25c6fc39 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:14.873684 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac253a78-e32f-4353-ba88-4bab25c6fc39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:14.875071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac253a78-e32f-4353-ba88-4bab25c6fc39 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:14.883368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 226/446] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:14.889314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac253a78-e32f-4353-ba88-4bab25c6fc39 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:14.905722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:14.962356 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733330e2-ceac-484f-8ecb-3b5a7ce6768e None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:15.003756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:15.006645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac253a78-e32f-4353-ba88-4bab25c6fc39 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:15.009009 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cd80f-af5d-4b3b-ba62-3cd7f6e664c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:15.009724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cd80f-af5d-4b3b-ba62-3cd7f6e664c5 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:15.024432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cd80f-af5d-4b3b-ba62-3cd7f6e664c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:15.024432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cd80f-af5d-4b3b-ba62-3cd7f6e664c5 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:15.024432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1cd80f-af5d-4b3b-ba62-3cd7f6e664c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:15.024432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1cd80f-af5d-4b3b-ba62-3cd7f6e664c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:15.024432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1cd80f-af5d-4b3b-ba62-3cd7f6e664c5 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:15.024432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 221/447] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:15.027404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cd80f-af5d-4b3b-ba62-3cd7f6e664c5 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:15.046977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ec6b8-9d16-4f37-a5ad-3870a15ef27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:15.047605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ec6b8-9d16-4f37-a5ad-3870a15ef27e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:15.047908 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ec6b8-9d16-4f37-a5ad-3870a15ef27e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:15.048163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ec6b8-9d16-4f37-a5ad-3870a15ef27e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:15.125832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ec6b8-9d16-4f37-a5ad-3870a15ef27e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:15.127450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ec6b8-9d16-4f37-a5ad-3870a15ef27e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:15.141359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cd80f-af5d-4b3b-ba62-3cd7f6e664c5 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:15.151156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:15.174637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d202d15-a4ca-4cc0-a67a-bec1d67d6bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:15.175290 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d202d15-a4ca-4cc0-a67a-bec1d67d6bfa None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:15.175566 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d202d15-a4ca-4cc0-a67a-bec1d67d6bfa None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:15.175838 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d202d15-a4ca-4cc0-a67a-bec1d67d6bfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:15.220583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:15.224162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 227/448] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 14:38:15.229962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ec6b8-9d16-4f37-a5ad-3870a15ef27e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:15.244589 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d202d15-a4ca-4cc0-a67a-bec1d67d6bfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:15.246057 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d202d15-a4ca-4cc0-a67a-bec1d67d6bfa None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:15.328246 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ec6b8-9d16-4f37-a5ad-3870a15ef27e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:15.346925 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 222/449] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 14:38:15.352641 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d202d15-a4ca-4cc0-a67a-bec1d67d6bfa None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:15.363267 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:15.363398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadda90d-5961-4d7c-ada1-d139f2833404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:15.364346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadda90d-5961-4d7c-ada1-d139f2833404 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:15.364818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadda90d-5961-4d7c-ada1-d139f2833404 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:15.365185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadda90d-5961-4d7c-ada1-d139f2833404 None admin] PATH_INFO: `/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:15.366304 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadda90d-5961-4d7c-ada1-d139f2833404 None admin] RBAC: Authorizing `identity:get_project(project_id=2a93ba0cc6a4409fa856a0a2fd7bfaa8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:15.367785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadda90d-5961-4d7c-ada1-d139f2833404 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:15.369416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 228/450] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:38:15 2022] GET /identity/v3/projects/2a93ba0cc6a4409fa856a0a2fd7bfaa8 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:15.375369 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadda90d-5961-4d7c-ada1-d139f2833404 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:15.440889 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:15.450441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d202d15-a4ca-4cc0-a67a-bec1d67d6bfa None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:15.473248 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadda90d-5961-4d7c-ada1-d139f2833404 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:15.483538 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406d2c3-0245-4337-86fa-daecf4a0e034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:15.484159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406d2c3-0245-4337-86fa-daecf4a0e034 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:15.484439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406d2c3-0245-4337-86fa-daecf4a0e034 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:15.484711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406d2c3-0245-4337-86fa-daecf4a0e034 None admin] PATH_INFO: `/v3/projects/d707a58a86f74ccbb5ffb499d9409139` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:15.485829 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406d2c3-0245-4337-86fa-daecf4a0e034 None admin] RBAC: Authorizing `identity:get_project(project_id=d707a58a86f74ccbb5ffb499d9409139)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:15.487295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406d2c3-0245-4337-86fa-daecf4a0e034 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:15.488906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 223/451] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:38:15 2022] GET /identity/v3/projects/d707a58a86f74ccbb5ffb499d9409139 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:15.494302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406d2c3-0245-4337-86fa-daecf4a0e034 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:15.508568 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eef817-4414-450f-b326-fee3f880c899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:15.509192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eef817-4414-450f-b326-fee3f880c899 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:15.509473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eef817-4414-450f-b326-fee3f880c899 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:15.509746 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eef817-4414-450f-b326-fee3f880c899 None admin] PATH_INFO: `/v3/groups/fb4974a00d9c4982aee946511fde7fee` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:15.510861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eef817-4414-450f-b326-fee3f880c899 None admin] RBAC: Authorizing `identity:get_group(group_id=fb4974a00d9c4982aee946511fde7fee)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:15.512888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eef817-4414-450f-b326-fee3f880c899 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:15.514449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 229/452] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 30 14:38:15 2022] GET /identity/v3/groups/fb4974a00d9c4982aee946511fde7fee => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:15.520040 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eef817-4414-450f-b326-fee3f880c899 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:15.570926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:15.591113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406d2c3-0245-4337-86fa-daecf4a0e034 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:15.615733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eef817-4414-450f-b326-fee3f880c899 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:15.626038 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da45b1-a848-48e8-ab01-924f96435581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:15.626643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da45b1-a848-48e8-ab01-924f96435581 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:15.626925 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da45b1-a848-48e8-ab01-924f96435581 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:15.627217 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da45b1-a848-48e8-ab01-924f96435581 None admin] PATH_INFO: `/v3/groups/73093109afa4403cb77794c287978bda` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:15.628319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da45b1-a848-48e8-ab01-924f96435581 None admin] RBAC: Authorizing `identity:get_group(group_id=73093109afa4403cb77794c287978bda)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:15.629730 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da45b1-a848-48e8-ab01-924f96435581 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:15.631296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 224/453] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Aug 30 14:38:15 2022] GET /identity/v3/groups/73093109afa4403cb77794c287978bda => generated 229 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:15.636870 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da45b1-a848-48e8-ab01-924f96435581 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:15.650149 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12530bdd-e4f8-449d-96e8-01b2231312d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:15.650745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12530bdd-e4f8-449d-96e8-01b2231312d8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:15.651028 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12530bdd-e4f8-449d-96e8-01b2231312d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:15.651344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12530bdd-e4f8-449d-96e8-01b2231312d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:15.652331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12530bdd-e4f8-449d-96e8-01b2231312d8 None admin] RBAC: Adding query filter params (group.id=fb4974a00d9c4982aee946511fde7fee, role.id=6fd5089acea448219d4c0a9a9424a3af, scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:15.652638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12530bdd-e4f8-449d-96e8-01b2231312d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:15.654063 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12530bdd-e4f8-449d-96e8-01b2231312d8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:15.656784 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:15.660001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 230/454] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 30 14:38:15 2022] GET /identity/v3/role_assignments?group.id=fb4974a00d9c4982aee946511fde7fee&scope.project.id=2a93ba0cc6a4409fa856a0a2fd7bfaa8&role.id=6fd5089acea448219d4c0a9a9424a3af => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:15.743125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da45b1-a848-48e8-ab01-924f96435581 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:15.778318 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a8e6f-63d7-4385-8416-3419f44bdb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:15.778846 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a8e6f-63d7-4385-8416-3419f44bdb39 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:15.779149 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a8e6f-63d7-4385-8416-3419f44bdb39 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:15.779428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a8e6f-63d7-4385-8416-3419f44bdb39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:15.780170 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a8e6f-63d7-4385-8416-3419f44bdb39 None admin] RBAC: Adding query filter params (group.id=73093109afa4403cb77794c287978bda, role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=d707a58a86f74ccbb5ffb499d9409139) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:15.780481 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a8e6f-63d7-4385-8416-3419f44bdb39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:15.781935 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a8e6f-63d7-4385-8416-3419f44bdb39 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:15.788249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 225/455] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Aug 30 14:38:15 2022] GET /identity/v3/role_assignments?group.id=73093109afa4403cb77794c287978bda&scope.project.id=d707a58a86f74ccbb5ffb499d9409139&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:18.242485 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e70a3-cf08-4b30-9c05-7467abbd0ac4 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:18.242995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e70a3-cf08-4b30-9c05-7467abbd0ac4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:18.242995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e70a3-cf08-4b30-9c05-7467abbd0ac4 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:18.244630 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 231/456] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:18.249993 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb453d5-58f8-4f4b-ac75-4d736b4ddb23 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:18.250826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb453d5-58f8-4f4b-ac75-4d736b4ddb23 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:18.251329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb453d5-58f8-4f4b-ac75-4d736b4ddb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:18.272213 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6d163-7865-4940-9ef8-eba14bae41f1 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:18.274089 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6d163-7865-4940-9ef8-eba14bae41f1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:18.274089 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6d163-7865-4940-9ef8-eba14bae41f1 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:18.274952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 232/457] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:18.278618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-cdb453d5-58f8-4f4b-ac75-4d736b4ddb23 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:18.279801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-cdb453d5-58f8-4f4b-ac75-4d736b4ddb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:18.280990 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675be10a-a433-4386-814d-07a7f57f9dcc None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:18.281477 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675be10a-a433-4386-814d-07a7f57f9dcc None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:18.281951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675be10a-a433-4386-814d-07a7f57f9dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:18.283533 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:18.310338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-675be10a-a433-4386-814d-07a7f57f9dcc None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:18.312717 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-675be10a-a433-4386-814d-07a7f57f9dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:18.408438 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 226/458] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:18.442030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 233/459] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:38:18.453834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecba25-a4d4-439b-8602-6ea566e293ef None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:18.454125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecba25-a4d4-439b-8602-6ea566e293ef None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:18.454403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecba25-a4d4-439b-8602-6ea566e293ef None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:18.455892 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 227/460] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:18.460965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfe929-11cf-45cc-a55f-9826f4984de4 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:18.461257 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfe929-11cf-45cc-a55f-9826f4984de4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:18.461530 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfe929-11cf-45cc-a55f-9826f4984de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:18.479896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-fdbfe929-11cf-45cc-a55f-9826f4984de4 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:18.481146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fdbfe929-11cf-45cc-a55f-9826f4984de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:18.564957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 234/461] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:38:18.749570 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecba25-a4d4-439b-8602-6ea566e293ef None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:18.749758 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfe929-11cf-45cc-a55f-9826f4984de4 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:18.848067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:18.898136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecba25-a4d4-439b-8602-6ea566e293ef None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:18.909882 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfe929-11cf-45cc-a55f-9826f4984de4 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:18.946187 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdec548-56c5-47e5-a93c-acca44ff13db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:18.946816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdec548-56c5-47e5-a93c-acca44ff13db None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:18.947114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdec548-56c5-47e5-a93c-acca44ff13db None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:18.947389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdec548-56c5-47e5-a93c-acca44ff13db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:18.954930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c921610-2af0-46e2-94eb-9a30828ee316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:18.955589 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c921610-2af0-46e2-94eb-9a30828ee316 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:18.955871 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c921610-2af0-46e2-94eb-9a30828ee316 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:18.956144 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c921610-2af0-46e2-94eb-9a30828ee316 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:18.956673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:18.959648 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdec548-56c5-47e5-a93c-acca44ff13db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:18.961110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdec548-56c5-47e5-a93c-acca44ff13db None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:18.966362 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c921610-2af0-46e2-94eb-9a30828ee316 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:18.967832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c921610-2af0-46e2-94eb-9a30828ee316 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:18.976255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-2fdec548-56c5-47e5-a93c-acca44ff13db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:18.976255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:18.976255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:18.976255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:18.976255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:18.976255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:18.976255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:18.976255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:18.976255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:18.976255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:18.976255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:18.976255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:18.976255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:18.976255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:18.976255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:18.976255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:18.984861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:18.985743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:18.985743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:18.985743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:18.985743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 228/462] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:18.985743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39334d-7711-4c74-ba84-11f76aecdd39 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:18.985743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39334d-7711-4c74-ba84-11f76aecdd39 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:18.985743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39334d-7711-4c74-ba84-11f76aecdd39 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:18.985743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 229/463] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:18.985743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39334d-7711-4c74-ba84-11f76aecdd39 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:18.988972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-0c921610-2af0-46e2-94eb-9a30828ee316 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:18.988972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:18.988972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:18.988972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:18.988972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:18.988972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:18.988972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:18.988972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:18.988972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:18.988972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:18.988972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:18.988972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:18.988972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:18.988972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:18.988972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:18.988972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:18.989817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:18.990655 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:18.990655 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:18.990655 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:18.990655 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 235/464] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:38:18.990881 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c921610-2af0-46e2-94eb-9a30828ee316 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:19.093960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c921610-2af0-46e2-94eb-9a30828ee316 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:19.094284 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39334d-7711-4c74-ba84-11f76aecdd39 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:19.106533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:19.127226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a206420-7d95-4442-a54b-acda02eebb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:19.127831 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a206420-7d95-4442-a54b-acda02eebb4a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:19.128154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a206420-7d95-4442-a54b-acda02eebb4a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:19.128612 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a206420-7d95-4442-a54b-acda02eebb4a None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:19.129444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a206420-7d95-4442-a54b-acda02eebb4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:19.129739 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a206420-7d95-4442-a54b-acda02eebb4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:19.130286 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c5be6-bd74-4697-a16b-05a260dd81af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:19.130910 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c5be6-bd74-4697-a16b-05a260dd81af None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:19.131215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c5be6-bd74-4697-a16b-05a260dd81af None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:19.131323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a206420-7d95-4442-a54b-acda02eebb4a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:19.131482 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c5be6-bd74-4697-a16b-05a260dd81af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:19.142140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c5be6-bd74-4697-a16b-05a260dd81af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:19.143579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c5be6-bd74-4697-a16b-05a260dd81af None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:19.149251 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 236/465] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:19.151542 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d056869-67e7-4df0-a655-c8015a949cf3 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:19.152038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d056869-67e7-4df0-a655-c8015a949cf3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:19.152918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d056869-67e7-4df0-a655-c8015a949cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:19.158695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-da0c5be6-bd74-4697-a16b-05a260dd81af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:19.158695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:19.158695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:19.158695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:19.158695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:19.158695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:19.158695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:19.158695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:19.158695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:19.158695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:19.158695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:19.158695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:19.158695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:19.158695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:19.158695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:19.158695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:19.159821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:19.160669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:19.160669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:19.160669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:19.160669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 230/466] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:19.160669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c5be6-bd74-4697-a16b-05a260dd81af None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:19.175798 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5d056869-67e7-4df0-a655-c8015a949cf3 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:19.177110 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5d056869-67e7-4df0-a655-c8015a949cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:19.210141 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:19.268440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 237/467] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:19.268440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d056869-67e7-4df0-a655-c8015a949cf3 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:19.286580 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c5be6-bd74-4697-a16b-05a260dd81af None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:19.334590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180717b-f5dd-47be-ab82-96c13e328f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:19.335578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180717b-f5dd-47be-ab82-96c13e328f31 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:19.335925 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180717b-f5dd-47be-ab82-96c13e328f31 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:19.336913 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180717b-f5dd-47be-ab82-96c13e328f31 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:19.337641 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180717b-f5dd-47be-ab82-96c13e328f31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:19.338232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180717b-f5dd-47be-ab82-96c13e328f31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:19.340005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180717b-f5dd-47be-ab82-96c13e328f31 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:19.357439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 231/468] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:19.358211 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180717b-f5dd-47be-ab82-96c13e328f31 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:19.376104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d056869-67e7-4df0-a655-c8015a949cf3 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:19.377839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:19.413398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b746d45-8c87-47de-80ad-18ea2ab02958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:19.414006 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b746d45-8c87-47de-80ad-18ea2ab02958 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:19.414284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b746d45-8c87-47de-80ad-18ea2ab02958 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:19.414555 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b746d45-8c87-47de-80ad-18ea2ab02958 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:19.415239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b746d45-8c87-47de-80ad-18ea2ab02958 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:19.416749 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b746d45-8c87-47de-80ad-18ea2ab02958 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:19.478320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180717b-f5dd-47be-ab82-96c13e328f31 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:19.482045 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:19.503587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 238/469] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:38:19 2022] POST /identity/v3/users => generated 246 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:38:19.504666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b746d45-8c87-47de-80ad-18ea2ab02958 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:19.527941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df9560-bea4-43cb-b729-6ca6f5da3dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:19.529380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df9560-bea4-43cb-b729-6ca6f5da3dc4 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:19.529877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df9560-bea4-43cb-b729-6ca6f5da3dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:19.530401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df9560-bea4-43cb-b729-6ca6f5da3dc4 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:19.531314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df9560-bea4-43cb-b729-6ca6f5da3dc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:19.531821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df9560-bea4-43cb-b729-6ca6f5da3dc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:19.533772 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df9560-bea4-43cb-b729-6ca6f5da3dc4 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:19.551058 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 232/470] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:19.556114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df9560-bea4-43cb-b729-6ca6f5da3dc4 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:19.667915 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b746d45-8c87-47de-80ad-18ea2ab02958 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:19.684839 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df9560-bea4-43cb-b729-6ca6f5da3dc4 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:19.707183 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d565f-c652-479b-a088-1adc92665e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:19.707792 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d565f-c652-479b-a088-1adc92665e6e None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:19.708070 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d565f-c652-479b-a088-1adc92665e6e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:19.708338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d565f-c652-479b-a088-1adc92665e6e None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:19.708988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d565f-c652-479b-a088-1adc92665e6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:19.710428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d565f-c652-479b-a088-1adc92665e6e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:19.723330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3acd84-9a82-47c2-8da4-a43344fc9310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:19.724082 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3acd84-9a82-47c2-8da4-a43344fc9310 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:19.724936 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3acd84-9a82-47c2-8da4-a43344fc9310 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:19.724936 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3acd84-9a82-47c2-8da4-a43344fc9310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:19.737265 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3acd84-9a82-47c2-8da4-a43344fc9310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:19.738116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3acd84-9a82-47c2-8da4-a43344fc9310 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:19.753958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-cf3acd84-9a82-47c2-8da4-a43344fc9310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:19.753958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:19.753958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:19.753958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:19.753958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:19.753958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:19.753958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:19.753958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:19.753958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:19.753958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:19.753958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:19.753958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:19.753958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:19.753958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:19.753958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:19.753958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:19.755019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:19.762582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:19.762582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:19.762582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:19.762582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 233/471] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:19.762582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3acd84-9a82-47c2-8da4-a43344fc9310 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:19.774628 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:19.827461 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 239/472] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:38:19 2022] POST /identity/v3/users => generated 248 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:38:19.828505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d565f-c652-479b-a088-1adc92665e6e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:19.855164 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:19.870133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3acd84-9a82-47c2-8da4-a43344fc9310 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:19.912755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79f477-4684-4994-b955-629671bdd8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:19.913452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79f477-4684-4994-b955-629671bdd8df None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:19.913731 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79f477-4684-4994-b955-629671bdd8df None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:19.914048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79f477-4684-4994-b955-629671bdd8df None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:19.914711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79f477-4684-4994-b955-629671bdd8df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:19.916201 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79f477-4684-4994-b955-629671bdd8df None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:19.951350 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d565f-c652-479b-a088-1adc92665e6e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:19.990664 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369f699-2c08-4431-a244-2f0aede7fe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:19.992146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369f699-2c08-4431-a244-2f0aede7fe74 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:19.992146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369f699-2c08-4431-a244-2f0aede7fe74 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:19.992146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369f699-2c08-4431-a244-2f0aede7fe74 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:19.992937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369f699-2c08-4431-a244-2f0aede7fe74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:19.993248 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369f699-2c08-4431-a244-2f0aede7fe74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:19.994737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369f699-2c08-4431-a244-2f0aede7fe74 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:20.006502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 240/473] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:20.007725 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b5624-d6c3-4a88-bc02-36673520f4a1 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:20.008011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b5624-d6c3-4a88-bc02-36673520f4a1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:20.008283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b5624-d6c3-4a88-bc02-36673520f4a1 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:20.009610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 241/474] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:20.013240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b5624-d6c3-4a88-bc02-36673520f4a1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:20.024203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 234/475] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:38:19 2022] POST /identity/v3/users => generated 248 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:38:20.025504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91c948-2b95-4605-a287-96239a96da59 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:20.029021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91c948-2b95-4605-a287-96239a96da59 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:20.029021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91c948-2b95-4605-a287-96239a96da59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:20.048573 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-ef91c948-2b95-4605-a287-96239a96da59 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:20.050708 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ef91c948-2b95-4605-a287-96239a96da59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:20.123898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:20.159431 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b5624-d6c3-4a88-bc02-36673520f4a1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:20.172735 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 235/476] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:20.232316 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bddb37-be9b-40f8-93dc-d91602a6d23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:20.233091 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bddb37-be9b-40f8-93dc-d91602a6d23a None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:20.233479 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bddb37-be9b-40f8-93dc-d91602a6d23a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:20.233855 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bddb37-be9b-40f8-93dc-d91602a6d23a None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:20.234623 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bddb37-be9b-40f8-93dc-d91602a6d23a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:20.236281 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bddb37-be9b-40f8-93dc-d91602a6d23a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:20.302423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 242/477] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:38:20 2022] POST /identity/v3/users => generated 249 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:38:20.490285 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91c948-2b95-4605-a287-96239a96da59 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:20.492380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:20.595513 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91c948-2b95-4605-a287-96239a96da59 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:20.630710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6cc210-31eb-4e3c-954b-69156918a2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:20.631709 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cc210-31eb-4e3c-954b-69156918a2f4 None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:20.632186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cc210-31eb-4e3c-954b-69156918a2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:20.632650 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cc210-31eb-4e3c-954b-69156918a2f4 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:20.633646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6cc210-31eb-4e3c-954b-69156918a2f4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:20.635237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6cc210-31eb-4e3c-954b-69156918a2f4 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:20.666638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.sql.core [None req-aa6cc210-31eb-4e3c-954b-69156918a2f4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 14:38:20.666638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 14:38:20.666638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [parameters: {'id': '990e646f8d1e4977b5ce8cb0fb12db50', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 14:38:20.666638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88058) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 14:38:20.683310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-aa6cc210-31eb-4e3c-954b-69156918a2f4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 14:38:20.683310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:20.683310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:38:20.683310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:38:20.683310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:38:20.683310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:38:20.683310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:38:20.683310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:38:20.683310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:38:20.683310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:38:20.683310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:38:20.683310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:38:20.683310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:38:20.683310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application result.read() Aug 30 14:38:20.683310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return ref.to_dict() Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:38:20.684704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application yield resource Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise self.value Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application session.commit() Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 14:38:20.685588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self.session.flush() Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application compat.raise_( Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 14:38:20.686478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 14:38:20.688348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 14:38:20.688348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 14:38:20.688348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 14:38:20.688348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 14:38:20.688348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 14:38:20.688348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 14:38:20.688348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 14:38:20.688348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 14:38:20.688348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 14:38:20.688348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 14:38:20.688348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 14:38:20.688348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 14:38:20.688348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 14:38:20.688348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 14:38:20.688348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:38:20.688348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception Aug 30 14:38:20.690298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:38:20.690298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:38:20.690298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:38:20.690298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:38:20.690298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:38:20.690298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:38:20.690298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:38:20.690298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:38:20.690298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:38:20.690298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:38:20.690298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:38:20.690298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application result.read() Aug 30 14:38:20.690298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:38:20.690298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:38:20.690298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:38:20.690298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:38:20.692402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:38:20.692402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:38:20.692402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:38:20.692402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:38:20.692402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 14:38:20.692402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 14:38:20.692402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [parameters: {'id': '990e646f8d1e4977b5ce8cb0fb12db50', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 14:38:20.692402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 14:38:20.692402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:20.692402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:38:20.692402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:20.692402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:20.692402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:20.692402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:20.692402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:20.692402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:20.694312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:20.694312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:20.694312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:20.694312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:20.694312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:20.694312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:20.694312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 30 14:38:20.694312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 30 14:38:20.694312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:20.694312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:20.694312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 30 14:38:20.694312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 30 14:38:20.694312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 14:38:20.694312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 14:38:20.694312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 14:38:20.694312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:20.703260 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 236/478] 10.222.0.27 () {60 vars in 1131 bytes} [Tue Aug 30 14:38:20 2022] POST /identity/v3/roles => generated 144 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:20.704880 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bddb37-be9b-40f8-93dc-d91602a6d23a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:20.822236 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bddb37-be9b-40f8-93dc-d91602a6d23a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:20.859197 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:20.865627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee6edf-dd84-426a-ad58-aea16c5dd5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:20.866420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee6edf-dd84-426a-ad58-aea16c5dd5db None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:20.866708 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee6edf-dd84-426a-ad58-aea16c5dd5db None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:20.866980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee6edf-dd84-426a-ad58-aea16c5dd5db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:20.883038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee6edf-dd84-426a-ad58-aea16c5dd5db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:20.887955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-64ee6edf-dd84-426a-ad58-aea16c5dd5db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:38:20.887955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:20.887955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:20.887955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:20.887955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:20.887955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:20.887955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:20.887955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:20.887955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:20.887955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:20.887955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:20.887955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:20.887955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:20.887955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:20.887955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:20.887955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:20.890050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:20.890050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:20.890050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:20.890050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:20.890050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:20.890050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:20.890050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:20.890050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:20.890050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:20.890050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:20.890050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:20.890050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:20.890050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:20.890050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:20.890050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:20.890050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:20.892067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:20.892067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:20.892067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:20.892067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:20.892067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:20.892067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:20.892067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:20.892067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:20.892067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:20.892067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:20.892067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:38:20.892067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:20.892067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 243/479] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:38:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:20.899411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6cc210-31eb-4e3c-954b-69156918a2f4 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:21.010536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:21.050643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6cc210-31eb-4e3c-954b-69156918a2f4 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:21.096691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81437e0d-3fc3-46f1-811d-42e000f147bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:21.097215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81437e0d-3fc3-46f1-811d-42e000f147bc None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:21.097497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81437e0d-3fc3-46f1-811d-42e000f147bc None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:21.097774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81437e0d-3fc3-46f1-811d-42e000f147bc None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:21.098452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81437e0d-3fc3-46f1-811d-42e000f147bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:21.098758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81437e0d-3fc3-46f1-811d-42e000f147bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:21.100160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81437e0d-3fc3-46f1-811d-42e000f147bc None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:21.114958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 237/480] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:21.741934 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5670b-bbf8-4acf-bf06-9d99fd829f56 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:21.742441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5670b-bbf8-4acf-bf06-9d99fd829f56 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:21.742513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5670b-bbf8-4acf-bf06-9d99fd829f56 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:21.744092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 244/481] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:21.750750 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4018a6d-4696-4a40-8a75-ee0fdffff7c4 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:21.751213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4018a6d-4696-4a40-8a75-ee0fdffff7c4 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:21.751390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4018a6d-4696-4a40-8a75-ee0fdffff7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:21.771790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-d4018a6d-4696-4a40-8a75-ee0fdffff7c4 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:21.773001 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d4018a6d-4696-4a40-8a75-ee0fdffff7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:21.850593 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 238/482] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:22.141868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5670b-bbf8-4acf-bf06-9d99fd829f56 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:22.153932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e68fd4-7073-4afe-b6b6-b82c290e21b7 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:22.154215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e68fd4-7073-4afe-b6b6-b82c290e21b7 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:22.156361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e68fd4-7073-4afe-b6b6-b82c290e21b7 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:22.162245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 239/483] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:22.163950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39b5c-32b5-40b9-8833-725b90646f92 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:22.164235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39b5c-32b5-40b9-8833-725b90646f92 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:22.164503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39b5c-32b5-40b9-8833-725b90646f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:22.193740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-dec39b5c-32b5-40b9-8833-725b90646f92 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:22.195257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-dec39b5c-32b5-40b9-8833-725b90646f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:22.284392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:22.312471 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 240/484] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:22.330095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5670b-bbf8-4acf-bf06-9d99fd829f56 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:22.387800 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cd7d8-ff77-4f6c-96d9-1beeccc1df83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:22.388439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cd7d8-ff77-4f6c-96d9-1beeccc1df83 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:22.388719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cd7d8-ff77-4f6c-96d9-1beeccc1df83 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:22.388990 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cd7d8-ff77-4f6c-96d9-1beeccc1df83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:22.403019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7cd7d8-ff77-4f6c-96d9-1beeccc1df83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:22.408202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-6e7cd7d8-ff77-4f6c-96d9-1beeccc1df83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:22.408202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:22.408202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:22.408202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:22.408202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:22.408202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:22.408202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:22.408202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:22.408202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:22.408202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:22.408202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:22.408202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:22.408202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:22.408202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:22.408202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:22.408202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:22.409075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:22.409075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:22.409075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:22.409075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:22.409075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:22.409075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:22.409075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:22.409075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:22.409075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:22.409075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:22.409075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:22.409075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:22.409075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:22.409075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:22.409075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:22.409075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:22.409861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:22.409861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:22.409861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:22.409861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:22.409861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:22.409861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:22.409861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:22.409861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:22.409861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:22.409861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:22.409861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:22.409861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:22.409861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 245/485] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:22.415115 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec39b5c-32b5-40b9-8833-725b90646f92 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:22.473362 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dd023-2077-4b7f-9c8e-a349bdcb038d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:22.473698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dd023-2077-4b7f-9c8e-a349bdcb038d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:22.473927 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dd023-2077-4b7f-9c8e-a349bdcb038d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:22.475590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 246/486] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:38:22.481185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7c68e-0a66-4f59-95b3-fded413aa504 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:22.481472 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7c68e-0a66-4f59-95b3-fded413aa504 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:22.482243 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7c68e-0a66-4f59-95b3-fded413aa504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:22.517730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-68d7c68e-0a66-4f59-95b3-fded413aa504 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:22.520361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-68d7c68e-0a66-4f59-95b3-fded413aa504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:22.573097 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec39b5c-32b5-40b9-8833-725b90646f92 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:22.607715 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:22.640435 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5107e9-25d3-403c-856f-abcca9d89b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:22.641044 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5107e9-25d3-403c-856f-abcca9d89b43 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:22.641325 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5107e9-25d3-403c-856f-abcca9d89b43 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:22.641597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5107e9-25d3-403c-856f-abcca9d89b43 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:22.643069 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5107e9-25d3-403c-856f-abcca9d89b43 None admin] RBAC: Adding query filter params (name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:22.643407 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5107e9-25d3-403c-856f-abcca9d89b43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:22.645054 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5107e9-25d3-403c-856f-abcca9d89b43 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:22.645287 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 247/487] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:22.646142 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7c68e-0a66-4f59-95b3-fded413aa504 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:22.661011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 241/488] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:22.666518 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5107e9-25d3-403c-856f-abcca9d89b43 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:22.746459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:22.813024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7c68e-0a66-4f59-95b3-fded413aa504 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:22.826045 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5107e9-25d3-403c-856f-abcca9d89b43 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:22.872138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292a2e6-e59a-499c-bf4d-ff42704a368c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:22.872868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292a2e6-e59a-499c-bf4d-ff42704a368c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:22.873147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292a2e6-e59a-499c-bf4d-ff42704a368c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:22.873416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292a2e6-e59a-499c-bf4d-ff42704a368c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:22.884190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292a2e6-e59a-499c-bf4d-ff42704a368c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:22.885982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d36af2-d879-415e-ad43-c3364f79d9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:22.886916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d36af2-d879-415e-ad43-c3364f79d9c6 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:22.887415 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d36af2-d879-415e-ad43-c3364f79d9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:22.887885 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d36af2-d879-415e-ad43-c3364f79d9c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:22.891120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-2292a2e6-e59a-499c-bf4d-ff42704a368c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:22.891120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:22.891120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:22.891120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:22.891120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:22.891120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:22.891120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:22.891120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:22.891120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:22.891120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:22.891120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:22.891120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:22.891120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:22.891120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:22.891120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:22.891120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:22.892078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:22.892078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:22.892078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:22.892078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:22.892078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:22.892078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:22.892078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:22.892078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:22.892078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:22.892078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:22.892078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:22.892078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:22.892078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:22.892078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:22.892078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:22.892078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:22.892885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:22.892885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:22.892885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:22.892885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:22.892885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:22.892885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:22.892885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:22.892885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:22.892885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:22.892885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:22.892885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:22.892885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:22.895183 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 248/489] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:22.896427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afa031-baf3-41bc-9221-dc11d97ada96 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:22.896709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afa031-baf3-41bc-9221-dc11d97ada96 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:22.896979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afa031-baf3-41bc-9221-dc11d97ada96 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:22.898695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 249/490] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:22.900187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afa031-baf3-41bc-9221-dc11d97ada96 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:22.904465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d36af2-d879-415e-ad43-c3364f79d9c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:22.905926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d36af2-d879-415e-ad43-c3364f79d9c6 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:22.921346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-79d36af2-d879-415e-ad43-c3364f79d9c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:22.921346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:22.921346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:22.921346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:22.921346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:22.921346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:22.921346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:22.921346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:22.921346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:22.921346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:22.921346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:22.921346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:22.921346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:22.921346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:22.921346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:22.921346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:22.922269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:22.923132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:22.923132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:22.923132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:22.923132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 242/491] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:38:22.928639 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485b5b4-5389-49e1-8e5f-c48d08ecbceb None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:22.928923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485b5b4-5389-49e1-8e5f-c48d08ecbceb None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:22.929196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485b5b4-5389-49e1-8e5f-c48d08ecbceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:22.946797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:22.950213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-6485b5b4-5389-49e1-8e5f-c48d08ecbceb None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:22.951439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6485b5b4-5389-49e1-8e5f-c48d08ecbceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:23.018358 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afa031-baf3-41bc-9221-dc11d97ada96 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:23.026699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 243/492] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:23.028287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485b5b4-5389-49e1-8e5f-c48d08ecbceb None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:23.059271 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017448c-3121-405f-936a-41101f90ca27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:23.059878 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017448c-3121-405f-936a-41101f90ca27 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:23.060158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017448c-3121-405f-936a-41101f90ca27 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:23.060877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017448c-3121-405f-936a-41101f90ca27 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:23.061729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017448c-3121-405f-936a-41101f90ca27 None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:23.062177 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017448c-3121-405f-936a-41101f90ca27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:23.063888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017448c-3121-405f-936a-41101f90ca27 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:23.068072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:23.076656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 250/493] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:23.081878 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017448c-3121-405f-936a-41101f90ca27 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:23.172218 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485b5b4-5389-49e1-8e5f-c48d08ecbceb None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:23.195060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017448c-3121-405f-936a-41101f90ca27 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:23.196835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:23.225262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae91114-511b-4cfe-97d7-0401da4ab1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:23.226740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae91114-511b-4cfe-97d7-0401da4ab1ae None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:23.227203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae91114-511b-4cfe-97d7-0401da4ab1ae None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:23.227868 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae91114-511b-4cfe-97d7-0401da4ab1ae None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:23.228682 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae91114-511b-4cfe-97d7-0401da4ab1ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:23.229128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae91114-511b-4cfe-97d7-0401da4ab1ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:23.233580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb818b-40bf-4ee6-abc9-510a7509cf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:23.233580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb818b-40bf-4ee6-abc9-510a7509cf31 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:23.233580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb818b-40bf-4ee6-abc9-510a7509cf31 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:23.233580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb818b-40bf-4ee6-abc9-510a7509cf31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:23.236974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae91114-511b-4cfe-97d7-0401da4ab1ae None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:23.244405 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb818b-40bf-4ee6-abc9-510a7509cf31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:23.249453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-5cfb818b-40bf-4ee6-abc9-510a7509cf31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:23.249453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:23.249453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:23.249453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:23.249453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:23.249453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:23.249453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:23.249453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:23.249453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:23.249453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:23.249453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:23.249453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:23.249453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:23.249453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:23.249453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:23.249453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:23.250536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:23.250536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:23.250536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:23.250536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:23.250536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:23.250536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:23.250536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:23.250536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:23.250536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:23.250536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:23.250536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:23.250536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:23.250536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:23.250536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:23.250536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:23.250536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:23.251546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:23.251546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:23.251546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:23.251546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:23.251546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:23.251546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:23.251546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:23.251546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:23.251546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:23.251546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:23.251546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:23.251546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:23.255177 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 251/494] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:23.255177 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb818b-40bf-4ee6-abc9-510a7509cf31 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:23.258965 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 244/495] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:23.267149 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae91114-511b-4cfe-97d7-0401da4ab1ae None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:23.365430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb818b-40bf-4ee6-abc9-510a7509cf31 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:23.375479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:23.385214 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae91114-511b-4cfe-97d7-0401da4ab1ae None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:23.401491 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66018b5a-7229-42bb-b3bf-7b60f3d286e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:23.402117 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66018b5a-7229-42bb-b3bf-7b60f3d286e7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:23.402971 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66018b5a-7229-42bb-b3bf-7b60f3d286e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:23.403290 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66018b5a-7229-42bb-b3bf-7b60f3d286e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:23.415761 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66018b5a-7229-42bb-b3bf-7b60f3d286e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:23.417289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66018b5a-7229-42bb-b3bf-7b60f3d286e7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:23.425435 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe69415-c70e-40a8-83e3-b59c5ea019fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:23.425978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe69415-c70e-40a8-83e3-b59c5ea019fd None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:23.426265 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe69415-c70e-40a8-83e3-b59c5ea019fd None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:23.426523 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe69415-c70e-40a8-83e3-b59c5ea019fd None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:23.427208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe69415-c70e-40a8-83e3-b59c5ea019fd None admin] RBAC: Adding query filter params (name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:23.427507 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe69415-c70e-40a8-83e3-b59c5ea019fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:23.428856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe69415-c70e-40a8-83e3-b59c5ea019fd None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:23.432398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-66018b5a-7229-42bb-b3bf-7b60f3d286e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:23.432398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:23.432398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:23.432398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:23.432398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:23.432398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:23.432398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:23.432398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:23.432398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:23.432398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:23.432398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:23.432398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:23.432398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:23.432398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:23.432398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:23.432398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:23.433343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:23.434978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:23.434978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:23.434978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:23.434978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 252/496] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:23.435754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66018b5a-7229-42bb-b3bf-7b60f3d286e7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:23.447414 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 245/497] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:23.447414 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe69415-c70e-40a8-83e3-b59c5ea019fd None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:23.455997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:23.548427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66018b5a-7229-42bb-b3bf-7b60f3d286e7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:23.552898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe69415-c70e-40a8-83e3-b59c5ea019fd None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:23.586706 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462327aa-31e1-4e4d-9c82-0460704d429e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:23.587346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462327aa-31e1-4e4d-9c82-0460704d429e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:23.587626 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462327aa-31e1-4e4d-9c82-0460704d429e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:23.587724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90ee1d-9fbd-4350-91cc-da273cb7f12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:23.587951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462327aa-31e1-4e4d-9c82-0460704d429e None admin] PATH_INFO: `/v3/users/nova` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:23.588276 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90ee1d-9fbd-4350-91cc-da273cb7f12c None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:23.588555 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90ee1d-9fbd-4350-91cc-da273cb7f12c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:23.588827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90ee1d-9fbd-4350-91cc-da273cb7f12c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:23.595204 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462327aa-31e1-4e4d-9c82-0460704d429e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:23.596661 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462327aa-31e1-4e4d-9c82-0460704d429e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:23.599964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90ee1d-9fbd-4350-91cc-da273cb7f12c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:23.605231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-6c90ee1d-9fbd-4350-91cc-da273cb7f12c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:23.605231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:23.605231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:23.605231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:23.605231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:23.605231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:23.605231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:23.605231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:23.605231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:23.605231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:23.605231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:23.605231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:23.605231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:23.605231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:23.605231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:23.605231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:23.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:23.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:23.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:23.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:23.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:23.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:23.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:23.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:23.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:23.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:23.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:23.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:23.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:23.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:23.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:23.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:23.612136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:23.612136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:23.612136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:23.612136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:23.612136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:23.612136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:23.612136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:23.612136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:23.612136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:23.612136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:23.612136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:23.612136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:23.612136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 246/498] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:23.612136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90ee1d-9fbd-4350-91cc-da273cb7f12c None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:23.612847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-462327aa-31e1-4e4d-9c82-0460704d429e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:38:23.612847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:23.612847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:23.612847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:23.612847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:23.612847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:23.612847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:23.612847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:23.612847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:23.612847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:23.612847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:23.612847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:23.612847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:23.612847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:23.612847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:23.612847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:23.613656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:23.614515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:23.614515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:23.614515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:23.614515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:23.614515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:23.614515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:23.614515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:23.614515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:23.614515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:38:23.614515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:23.614515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 253/499] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 30 14:38:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:23.614515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462327aa-31e1-4e4d-9c82-0460704d429e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:23.633118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:23.694695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:23.721834 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462327aa-31e1-4e4d-9c82-0460704d429e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:23.722173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90ee1d-9fbd-4350-91cc-da273cb7f12c None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:23.758200 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfa735-011d-46eb-8c73-6b89715363f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:23.758790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfa735-011d-46eb-8c73-6b89715363f3 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:23.759063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfa735-011d-46eb-8c73-6b89715363f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:23.759356 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfa735-011d-46eb-8c73-6b89715363f3 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:23.759988 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbfa735-011d-46eb-8c73-6b89715363f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:23.760287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbfa735-011d-46eb-8c73-6b89715363f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:23.761176 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33807aba-0e38-4851-bd4f-4bd80d0ec613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:23.761669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbfa735-011d-46eb-8c73-6b89715363f3 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:23.762338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33807aba-0e38-4851-bd4f-4bd80d0ec613 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:23.762621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33807aba-0e38-4851-bd4f-4bd80d0ec613 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:23.762888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33807aba-0e38-4851-bd4f-4bd80d0ec613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:23.779061 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 247/500] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:23.779061 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfa735-011d-46eb-8c73-6b89715363f3 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:23.781018 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33807aba-0e38-4851-bd4f-4bd80d0ec613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:23.782483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33807aba-0e38-4851-bd4f-4bd80d0ec613 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:23.803636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-33807aba-0e38-4851-bd4f-4bd80d0ec613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:23.803636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:23.803636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:23.803636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:23.803636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:23.803636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:23.803636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:23.803636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:23.803636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:23.803636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:23.803636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:23.803636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:23.803636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:23.803636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:23.803636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:23.803636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:23.804560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:23.805394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:23.805394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:23.805394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:23.807427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 254/501] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:23.808780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33807aba-0e38-4851-bd4f-4bd80d0ec613 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:23.859261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:23.883647 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfa735-011d-46eb-8c73-6b89715363f3 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:23.909895 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33807aba-0e38-4851-bd4f-4bd80d0ec613 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:23.918409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7720693-4a2d-4e12-8f6e-db044d2b9523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:23.919011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7720693-4a2d-4e12-8f6e-db044d2b9523 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:23.919317 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7720693-4a2d-4e12-8f6e-db044d2b9523 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:23.919593 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7720693-4a2d-4e12-8f6e-db044d2b9523 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:23.920256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7720693-4a2d-4e12-8f6e-db044d2b9523 None admin] RBAC: Adding query filter params (name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:23.920558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7720693-4a2d-4e12-8f6e-db044d2b9523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:23.921920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7720693-4a2d-4e12-8f6e-db044d2b9523 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:23.929685 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 248/502] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:23.930938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7720693-4a2d-4e12-8f6e-db044d2b9523 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:23.938695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:23.948010 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b97c5-d151-4210-b43e-11be1ce44103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:23.948327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b97c5-d151-4210-b43e-11be1ce44103 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:23.948619 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b97c5-d151-4210-b43e-11be1ce44103 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:23.948895 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b97c5-d151-4210-b43e-11be1ce44103 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:23.950131 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b97c5-d151-4210-b43e-11be1ce44103 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:23.950435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b97c5-d151-4210-b43e-11be1ce44103 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:23.951902 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b97c5-d151-4210-b43e-11be1ce44103 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:23.987998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 255/503] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 30 14:38:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:23.989906 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b97c5-d151-4210-b43e-11be1ce44103 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:24.034237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7720693-4a2d-4e12-8f6e-db044d2b9523 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:24.067224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa51e4-1851-4946-b11a-89afc01cc4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:24.067846 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa51e4-1851-4946-b11a-89afc01cc4bd None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:24.068123 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa51e4-1851-4946-b11a-89afc01cc4bd None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:24.068394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa51e4-1851-4946-b11a-89afc01cc4bd None admin] PATH_INFO: `/v3/users/glance` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:24.071039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:24.077579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa51e4-1851-4946-b11a-89afc01cc4bd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:24.078965 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa51e4-1851-4946-b11a-89afc01cc4bd None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:24.088114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b97c5-d151-4210-b43e-11be1ce44103 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:24.092404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-dfaa51e4-1851-4946-b11a-89afc01cc4bd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:38:24.092404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:24.092404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:24.092404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:24.092404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:24.092404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:24.092404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:24.092404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:24.092404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:24.092404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:24.092404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:24.092404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:24.092404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:24.092404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:24.092404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:24.092404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:24.093492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:24.094389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:24.094389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:24.094389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:24.094389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:24.094389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:24.094389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:24.094389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:24.094389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:24.094389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:38:24.094389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:24.094389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 249/504] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:24.094946 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b349ecd-53a8-470d-9b21-9a829090a57a None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:24.095527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b349ecd-53a8-470d-9b21-9a829090a57a None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:24.095527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b349ecd-53a8-470d-9b21-9a829090a57a None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:24.096848 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 250/505] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:24.097392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b349ecd-53a8-470d-9b21-9a829090a57a None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:24.124605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79204bf6-e45d-4e1e-af81-1d197e27d6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:24.125207 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79204bf6-e45d-4e1e-af81-1d197e27d6c9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:24.125488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79204bf6-e45d-4e1e-af81-1d197e27d6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:24.125766 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79204bf6-e45d-4e1e-af81-1d197e27d6c9 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:24.126412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79204bf6-e45d-4e1e-af81-1d197e27d6c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:24.126720 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79204bf6-e45d-4e1e-af81-1d197e27d6c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:24.128147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79204bf6-e45d-4e1e-af81-1d197e27d6c9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:24.141599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 256/506] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:24.142729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79204bf6-e45d-4e1e-af81-1d197e27d6c9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:24.189188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:24.202419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b349ecd-53a8-470d-9b21-9a829090a57a None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:24.236380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325f46a-e074-44e9-a77a-64f96823af42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:24.237004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325f46a-e074-44e9-a77a-64f96823af42 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:24.237284 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325f46a-e074-44e9-a77a-64f96823af42 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:24.237554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325f46a-e074-44e9-a77a-64f96823af42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:24.243440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79204bf6-e45d-4e1e-af81-1d197e27d6c9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:24.248729 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325f46a-e074-44e9-a77a-64f96823af42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:24.250208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325f46a-e074-44e9-a77a-64f96823af42 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:24.265544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-f325f46a-e074-44e9-a77a-64f96823af42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:24.265544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:24.265544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:24.265544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:24.265544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:24.265544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:24.265544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:24.265544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:24.265544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:24.265544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:24.265544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:24.265544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:24.265544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:24.265544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:24.265544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:24.265544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:24.275227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:24.277137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:24.277137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:24.277137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:24.277137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 251/507] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:24.277137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6eee6-ea6d-430f-b8ee-2e350b35491e None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:24.277137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6eee6-ea6d-430f-b8ee-2e350b35491e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:24.277137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6eee6-ea6d-430f-b8ee-2e350b35491e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:24.277969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1f892-c3e8-4037-bfa5-e43f953b810c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:24.278626 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1f892-c3e8-4037-bfa5-e43f953b810c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:24.278854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1f892-c3e8-4037-bfa5-e43f953b810c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:24.279152 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1f892-c3e8-4037-bfa5-e43f953b810c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:24.288458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-1ac6eee6-ea6d-430f-b8ee-2e350b35491e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:24.289639 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1ac6eee6-ea6d-430f-b8ee-2e350b35491e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:24.331896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:24.347314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1f892-c3e8-4037-bfa5-e43f953b810c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:24.348873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1f892-c3e8-4037-bfa5-e43f953b810c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:24.355168 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 252/508] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:24.356560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6eee6-ea6d-430f-b8ee-2e350b35491e None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:24.406802 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:24.467052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 257/509] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:24.467609 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1f892-c3e8-4037-bfa5-e43f953b810c None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:24.478526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6eee6-ea6d-430f-b8ee-2e350b35491e None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:24.514522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5e7b7-1909-41f9-a354-62a9147ba61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:24.515488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5e7b7-1909-41f9-a354-62a9147ba61f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:24.515770 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5e7b7-1909-41f9-a354-62a9147ba61f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:24.516040 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5e7b7-1909-41f9-a354-62a9147ba61f None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:24.516856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5e7b7-1909-41f9-a354-62a9147ba61f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:24.517276 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5e7b7-1909-41f9-a354-62a9147ba61f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:24.518697 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5e7b7-1909-41f9-a354-62a9147ba61f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:24.562418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 253/510] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Aug 30 14:38:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:24.563155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5e7b7-1909-41f9-a354-62a9147ba61f None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:24.571353 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1f892-c3e8-4037-bfa5-e43f953b810c None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:24.597739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:24.607644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760df6a-9dbf-4549-830a-e6eb360a878f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:24.608274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760df6a-9dbf-4549-830a-e6eb360a878f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:24.608553 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760df6a-9dbf-4549-830a-e6eb360a878f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:24.608824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760df6a-9dbf-4549-830a-e6eb360a878f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:24.615773 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760df6a-9dbf-4549-830a-e6eb360a878f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:24.617237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760df6a-9dbf-4549-830a-e6eb360a878f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:24.628824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-9760df6a-9dbf-4549-830a-e6eb360a878f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:38:24.628824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:24.628824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:24.628824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:24.628824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:24.628824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:24.628824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:24.628824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:24.628824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:24.628824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:24.628824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:24.628824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:24.628824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:24.628824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:24.628824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:24.628824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:24.629752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:24.630641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:24.630641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:24.630641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:24.630641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:24.630641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:24.630641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:24.630641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:24.630641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:24.630641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:38:24.630641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:24.630641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 258/511] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:24.630641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760df6a-9dbf-4549-830a-e6eb360a878f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:24.634093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:24.666249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5e7b7-1909-41f9-a354-62a9147ba61f None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:24.699054 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3cf072-9134-4269-89aa-731ca5645077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:24.700192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3cf072-9134-4269-89aa-731ca5645077 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:24.700466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3cf072-9134-4269-89aa-731ca5645077 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:24.700733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3cf072-9134-4269-89aa-731ca5645077 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:24.701427 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3cf072-9134-4269-89aa-731ca5645077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:24.701727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3cf072-9134-4269-89aa-731ca5645077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:24.703139 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3cf072-9134-4269-89aa-731ca5645077 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:24.714110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 254/512] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:24.715467 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3cf072-9134-4269-89aa-731ca5645077 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:24.727849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760df6a-9dbf-4549-830a-e6eb360a878f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:24.762989 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef346d0b-6074-4171-a5b7-d90868744fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:24.763644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef346d0b-6074-4171-a5b7-d90868744fe9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:24.763927 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef346d0b-6074-4171-a5b7-d90868744fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:24.764204 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef346d0b-6074-4171-a5b7-d90868744fe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:24.775366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef346d0b-6074-4171-a5b7-d90868744fe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:24.777150 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef346d0b-6074-4171-a5b7-d90868744fe9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:24.791552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-ef346d0b-6074-4171-a5b7-d90868744fe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:24.791552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:24.791552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:24.791552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:24.791552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:24.791552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:24.791552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:24.791552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:24.791552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:24.791552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:24.791552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:24.791552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:24.791552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:24.791552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:24.791552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:24.791552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:24.792482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:24.793484 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:24.793484 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:24.793484 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:24.793484 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 259/513] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:24.793484 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef346d0b-6074-4171-a5b7-d90868744fe9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:24.813132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:24.818809 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3cf072-9134-4269-89aa-731ca5645077 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:24.852063 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:24.853382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b9be5-a3af-4615-92f5-8832fee136ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:24.853971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b9be5-a3af-4615-92f5-8832fee136ed None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:24.854246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b9be5-a3af-4615-92f5-8832fee136ed None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:24.854517 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b9be5-a3af-4615-92f5-8832fee136ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:24.895393 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef346d0b-6074-4171-a5b7-d90868744fe9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:24.920408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b9be5-a3af-4615-92f5-8832fee136ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:24.921842 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b9be5-a3af-4615-92f5-8832fee136ed None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:24.928395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71762dd3-2992-4d33-b956-a0969a0eba1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:24.929005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71762dd3-2992-4d33-b956-a0969a0eba1a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:24.929284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71762dd3-2992-4d33-b956-a0969a0eba1a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:24.929552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71762dd3-2992-4d33-b956-a0969a0eba1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:24.940170 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71762dd3-2992-4d33-b956-a0969a0eba1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:24.941591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71762dd3-2992-4d33-b956-a0969a0eba1a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:24.956695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-71762dd3-2992-4d33-b956-a0969a0eba1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:24.956695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:24.956695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:24.956695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:24.956695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:24.956695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:24.956695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:24.956695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:24.956695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:24.956695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:24.956695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:24.956695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:24.956695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:24.956695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:24.956695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:24.956695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:24.957603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:24.958427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:24.958427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:24.958427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:24.958427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 260/514] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:24.963403 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71762dd3-2992-4d33-b956-a0969a0eba1a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:25.013667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:25.019722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 255/515] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:25.020547 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b9be5-a3af-4615-92f5-8832fee136ed None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:25.062230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71762dd3-2992-4d33-b956-a0969a0eba1a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:25.070889 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:25.096079 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11816d4-a6e9-4959-9048-9d20648cf5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:25.096663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11816d4-a6e9-4959-9048-9d20648cf5d5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:25.096937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11816d4-a6e9-4959-9048-9d20648cf5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:25.097201 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11816d4-a6e9-4959-9048-9d20648cf5d5 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:25.097925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11816d4-a6e9-4959-9048-9d20648cf5d5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:25.098226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11816d4-a6e9-4959-9048-9d20648cf5d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:25.099711 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11816d4-a6e9-4959-9048-9d20648cf5d5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:25.117836 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 261/516] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Aug 30 14:38:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:25.118467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11816d4-a6e9-4959-9048-9d20648cf5d5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:25.120827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b9be5-a3af-4615-92f5-8832fee136ed None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:25.157278 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43ad67-c598-416e-acd3-617b9f8e5504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:25.157898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43ad67-c598-416e-acd3-617b9f8e5504 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:25.158178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43ad67-c598-416e-acd3-617b9f8e5504 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:25.158450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43ad67-c598-416e-acd3-617b9f8e5504 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:25.165317 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43ad67-c598-416e-acd3-617b9f8e5504 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:25.166763 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43ad67-c598-416e-acd3-617b9f8e5504 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:25.179725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-da43ad67-c598-416e-acd3-617b9f8e5504 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:38:25.179725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:25.179725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:25.179725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:25.179725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:25.179725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:25.179725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:25.179725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:25.179725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:25.179725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:25.179725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:25.179725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:25.179725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:25.179725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:25.179725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:25.179725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:25.180764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:25.181804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:25.181804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:25.181804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:25.181804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:25.181804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:25.181804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:25.181804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:25.181804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:25.181804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:38:25.181804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:25.181804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 256/517] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:25.181804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43ad67-c598-416e-acd3-617b9f8e5504 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:25.221875 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11816d4-a6e9-4959-9048-9d20648cf5d5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:25.237157 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:25.256342 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222b3f6-6d8f-481e-9c75-468847067ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:25.256934 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222b3f6-6d8f-481e-9c75-468847067ff7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:25.257210 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222b3f6-6d8f-481e-9c75-468847067ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:25.257497 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222b3f6-6d8f-481e-9c75-468847067ff7 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:25.258135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222b3f6-6d8f-481e-9c75-468847067ff7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:25.258435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222b3f6-6d8f-481e-9c75-468847067ff7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:25.259898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222b3f6-6d8f-481e-9c75-468847067ff7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:25.271696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 262/518] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:25.272511 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222b3f6-6d8f-481e-9c75-468847067ff7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:25.285357 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43ad67-c598-416e-acd3-617b9f8e5504 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:25.300509 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:25.318466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2bcf5-ec8c-4cbf-b391-c8906d0b4034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:25.319063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2bcf5-ec8c-4cbf-b391-c8906d0b4034 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:25.319448 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2bcf5-ec8c-4cbf-b391-c8906d0b4034 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:25.319641 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2bcf5-ec8c-4cbf-b391-c8906d0b4034 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:25.320278 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2bcf5-ec8c-4cbf-b391-c8906d0b4034 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:25.320584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2bcf5-ec8c-4cbf-b391-c8906d0b4034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:25.321946 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2bcf5-ec8c-4cbf-b391-c8906d0b4034 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:25.334103 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 257/519] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:25.334875 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2bcf5-ec8c-4cbf-b391-c8906d0b4034 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:25.377563 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222b3f6-6d8f-481e-9c75-468847067ff7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:25.411974 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1a3db-a611-4a25-a208-f2fbccc9dce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:25.412613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1a3db-a611-4a25-a208-f2fbccc9dce1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:25.412899 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1a3db-a611-4a25-a208-f2fbccc9dce1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:25.413172 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1a3db-a611-4a25-a208-f2fbccc9dce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:25.423288 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c1a3db-a611-4a25-a208-f2fbccc9dce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:25.424770 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c1a3db-a611-4a25-a208-f2fbccc9dce1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:25.439670 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2bcf5-ec8c-4cbf-b391-c8906d0b4034 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:25.439797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-96c1a3db-a611-4a25-a208-f2fbccc9dce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:25.439797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:25.439797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:25.439797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:25.439797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:25.439797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:25.439797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:25.439797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:25.439797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:25.439797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:25.439797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:25.439797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:25.439797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:25.439797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:25.439797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:25.439797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:25.440718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:25.441536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:25.441536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:25.441536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:25.441536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 263/520] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:25.441536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1a3db-a611-4a25-a208-f2fbccc9dce1 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:25.460846 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:25.478245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50dede5-825d-40a6-bec3-69383e3e110c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:25.478845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dede5-825d-40a6-bec3-69383e3e110c None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:25.479144 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dede5-825d-40a6-bec3-69383e3e110c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:25.479416 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50dede5-825d-40a6-bec3-69383e3e110c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:25.519943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:25.542315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1a3db-a611-4a25-a208-f2fbccc9dce1 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:25.549819 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50dede5-825d-40a6-bec3-69383e3e110c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:25.551310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50dede5-825d-40a6-bec3-69383e3e110c None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:25.575274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffe540-328b-4ff2-867e-eabf2a5be9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:25.575869 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffe540-328b-4ff2-867e-eabf2a5be9b9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:25.576142 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffe540-328b-4ff2-867e-eabf2a5be9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:25.576411 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffe540-328b-4ff2-867e-eabf2a5be9b9 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:25.577129 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffe540-328b-4ff2-867e-eabf2a5be9b9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:25.577428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffe540-328b-4ff2-867e-eabf2a5be9b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:25.578835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffe540-328b-4ff2-867e-eabf2a5be9b9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:25.595711 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 264/521] 10.222.0.27 () {56 vars in 1139 bytes} [Tue Aug 30 14:38:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:25.596486 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffe540-328b-4ff2-867e-eabf2a5be9b9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:25.648323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 258/522] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:25.649220 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50dede5-825d-40a6-bec3-69383e3e110c None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:25.667364 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:25.695344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffe540-328b-4ff2-867e-eabf2a5be9b9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:25.728234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff3cd4-66b8-4554-ba7c-fafb80960486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:25.728853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff3cd4-66b8-4554-ba7c-fafb80960486 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:25.729135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff3cd4-66b8-4554-ba7c-fafb80960486 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:25.729410 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff3cd4-66b8-4554-ba7c-fafb80960486 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:25.732039 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:25.736544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff3cd4-66b8-4554-ba7c-fafb80960486 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:25.737993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff3cd4-66b8-4554-ba7c-fafb80960486 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:25.747228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50dede5-825d-40a6-bec3-69383e3e110c None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:25.750104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-abff3cd4-66b8-4554-ba7c-fafb80960486 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:25.750104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:25.750104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:25.750104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:25.750104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:25.750104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:25.750104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:25.750104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:25.750104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:25.750104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:25.750104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:25.750104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:25.750104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:25.750104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:25.750104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:25.751081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:25.752156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:25.752156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:25.752156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:25.752156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:25.752156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:25.752156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:25.752156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:25.752156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:25.752156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 265/523] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:25.752698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff3cd4-66b8-4554-ba7c-fafb80960486 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:25.782407 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773daab-5b9d-4c5f-a568-d0dd7ad14980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:25.782938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773daab-5b9d-4c5f-a568-d0dd7ad14980 None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:25.783245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773daab-5b9d-4c5f-a568-d0dd7ad14980 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:25.783519 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773daab-5b9d-4c5f-a568-d0dd7ad14980 None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:25.784168 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4773daab-5b9d-4c5f-a568-d0dd7ad14980 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:25.785599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4773daab-5b9d-4c5f-a568-d0dd7ad14980 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:25.851454 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff3cd4-66b8-4554-ba7c-fafb80960486 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:25.866278 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 259/524] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:38:25 2022] POST /identity/v3/users => generated 247 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:38:25.867121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773daab-5b9d-4c5f-a568-d0dd7ad14980 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:25.897005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12240b-c1dd-4de9-addc-4f62910d4005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:25.897712 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12240b-c1dd-4de9-addc-4f62910d4005 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:25.897998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12240b-c1dd-4de9-addc-4f62910d4005 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:25.898253 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12240b-c1dd-4de9-addc-4f62910d4005 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:25.898951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12240b-c1dd-4de9-addc-4f62910d4005 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:25.899285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12240b-c1dd-4de9-addc-4f62910d4005 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:25.900815 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12240b-c1dd-4de9-addc-4f62910d4005 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:25.913910 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 266/525] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:25.914539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12240b-c1dd-4de9-addc-4f62910d4005 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:25.952580 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:25.980061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:25.985502 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773daab-5b9d-4c5f-a568-d0dd7ad14980 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:26.027567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa9f08-a5ab-4320-998a-1b7ed7074aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:26.028178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa9f08-a5ab-4320-998a-1b7ed7074aa0 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:26.028453 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa9f08-a5ab-4320-998a-1b7ed7074aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:26.028738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa9f08-a5ab-4320-998a-1b7ed7074aa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:26.032793 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12240b-c1dd-4de9-addc-4f62910d4005 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:26.075466 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0ec74-6ba9-42ea-84b5-24481b009da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:26.076337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0ec74-6ba9-42ea-84b5-24481b009da6 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:26.076756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0ec74-6ba9-42ea-84b5-24481b009da6 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:26.077162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0ec74-6ba9-42ea-84b5-24481b009da6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:26.092531 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0ec74-6ba9-42ea-84b5-24481b009da6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:26.094995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0ec74-6ba9-42ea-84b5-24481b009da6 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:26.106417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa9f08-a5ab-4320-998a-1b7ed7074aa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:26.107828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa9f08-a5ab-4320-998a-1b7ed7074aa0 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:26.112852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-3cd0ec74-6ba9-42ea-84b5-24481b009da6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:26.112852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:26.112852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:26.112852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:26.112852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:26.112852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:26.112852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:26.112852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:26.112852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:26.112852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:26.112852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:26.112852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:26.112852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:26.112852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:26.112852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:26.112852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:26.113929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:26.114809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:26.114809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:26.114809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:26.120268 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 267/526] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:26.120268 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0ec74-6ba9-42ea-84b5-24481b009da6 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:26.186428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:26.220452 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0ec74-6ba9-42ea-84b5-24481b009da6 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:26.221110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 260/527] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:26.221948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa9f08-a5ab-4320-998a-1b7ed7074aa0 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:26.237189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:26.269592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed4bf0-84e3-4d8b-b4a8-a60b6e2ce255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:26.270515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed4bf0-84e3-4d8b-b4a8-a60b6e2ce255 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:26.270793 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed4bf0-84e3-4d8b-b4a8-a60b6e2ce255 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:26.271182 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed4bf0-84e3-4d8b-b4a8-a60b6e2ce255 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:26.271825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed4bf0-84e3-4d8b-b4a8-a60b6e2ce255 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:26.272129 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed4bf0-84e3-4d8b-b4a8-a60b6e2ce255 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:26.273610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed4bf0-84e3-4d8b-b4a8-a60b6e2ce255 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:26.292543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 268/528] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:26.292543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed4bf0-84e3-4d8b-b4a8-a60b6e2ce255 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:26.343937 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa9f08-a5ab-4320-998a-1b7ed7074aa0 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:26.379167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0a18d-711d-4fb1-b21c-934e455bdda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:26.379789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0a18d-711d-4fb1-b21c-934e455bdda0 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:26.380062 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0a18d-711d-4fb1-b21c-934e455bdda0 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:26.380326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0a18d-711d-4fb1-b21c-934e455bdda0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:26.386855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0a18d-711d-4fb1-b21c-934e455bdda0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:26.388333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0a18d-711d-4fb1-b21c-934e455bdda0 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:26.399821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-3fb0a18d-711d-4fb1-b21c-934e455bdda0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:26.399821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:26.399821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:26.399821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:26.399821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:26.399821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:26.399821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:26.399821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:26.399821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:26.399821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:26.399821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:26.399821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:26.399821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:26.399821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:26.399821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:26.400676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:26.401789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:26.401789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:26.401789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:26.401789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:26.401789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:26.401789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:26.401789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:26.401789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:26.401789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 261/529] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:26.402767 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0a18d-711d-4fb1-b21c-934e455bdda0 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:26.408900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed4bf0-84e3-4d8b-b4a8-a60b6e2ce255 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:26.434614 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:26.445510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f47b47-c07a-46e6-934f-128a65347c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:26.446318 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f47b47-c07a-46e6-934f-128a65347c5e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:26.446649 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f47b47-c07a-46e6-934f-128a65347c5e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:26.446993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f47b47-c07a-46e6-934f-128a65347c5e None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:26.447691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f47b47-c07a-46e6-934f-128a65347c5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:26.448002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f47b47-c07a-46e6-934f-128a65347c5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:26.449471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f47b47-c07a-46e6-934f-128a65347c5e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:26.466668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 269/530] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:26.466668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f47b47-c07a-46e6-934f-128a65347c5e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:26.499021 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:26.505826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0a18d-711d-4fb1-b21c-934e455bdda0 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:26.541385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217aae0-32f6-483a-8314-3a30e9195bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:26.542008 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217aae0-32f6-483a-8314-3a30e9195bf5 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:26.542288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217aae0-32f6-483a-8314-3a30e9195bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:26.542608 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217aae0-32f6-483a-8314-3a30e9195bf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:26.554627 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217aae0-32f6-483a-8314-3a30e9195bf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:26.556111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217aae0-32f6-483a-8314-3a30e9195bf5 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:26.570348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-3217aae0-32f6-483a-8314-3a30e9195bf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:26.570348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:26.570348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:26.570348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:26.570348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:26.570348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:26.570348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:26.570348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:26.570348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:26.570348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:26.570348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:26.570348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:26.570348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:26.570348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:26.570348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:26.570348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:26.571263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:26.572090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:26.572090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:26.572090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:26.572090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 262/531] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:26.572323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217aae0-32f6-483a-8314-3a30e9195bf5 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:26.581771 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f47b47-c07a-46e6-934f-128a65347c5e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:26.617941 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66450511-677e-48bf-bf8d-d421ddd2214f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:26.618533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450511-677e-48bf-bf8d-d421ddd2214f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:26.618813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450511-677e-48bf-bf8d-d421ddd2214f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:26.619080 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450511-677e-48bf-bf8d-d421ddd2214f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:26.619842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66450511-677e-48bf-bf8d-d421ddd2214f None admin] RBAC: Adding query filter params (role.id=244ad93ecd1949c9a6e8f74b135b43b4, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=5b6e4c5bf2a54b28a55e5ae74b90ab4f) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:26.620158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66450511-677e-48bf-bf8d-d421ddd2214f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:26.621615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66450511-677e-48bf-bf8d-d421ddd2214f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:26.635135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 270/532] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:38:26 2022] GET /identity/v3/role_assignments?user.id=5b6e4c5bf2a54b28a55e5ae74b90ab4f&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=244ad93ecd1949c9a6e8f74b135b43b4 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:26.635135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66450511-677e-48bf-bf8d-d421ddd2214f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:26.674543 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:26.693262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217aae0-32f6-483a-8314-3a30e9195bf5 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:26.742575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c28f7-f413-4a58-8776-7cf9c4957762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:26.743224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c28f7-f413-4a58-8776-7cf9c4957762 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:26.743507 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c28f7-f413-4a58-8776-7cf9c4957762 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:26.743777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c28f7-f413-4a58-8776-7cf9c4957762 None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:26.745428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c28f7-f413-4a58-8776-7cf9c4957762 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:26.745746 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c28f7-f413-4a58-8776-7cf9c4957762 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:26.747838 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c28f7-f413-4a58-8776-7cf9c4957762 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:26.749043 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:26.758590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66450511-677e-48bf-bf8d-d421ddd2214f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:26.774462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 263/533] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:26.775497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c28f7-f413-4a58-8776-7cf9c4957762 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:26.806056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884ed4e-db7b-4417-a3a1-79c4a085c0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:26.806606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884ed4e-db7b-4417-a3a1-79c4a085c0ea None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:26.806885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884ed4e-db7b-4417-a3a1-79c4a085c0ea None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:26.807781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884ed4e-db7b-4417-a3a1-79c4a085c0ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:26.815230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884ed4e-db7b-4417-a3a1-79c4a085c0ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:26.816694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884ed4e-db7b-4417-a3a1-79c4a085c0ea None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:26.829634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-a884ed4e-db7b-4417-a3a1-79c4a085c0ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:26.829634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:26.829634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:26.829634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:26.829634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:26.829634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:26.829634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:26.829634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:26.829634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:26.829634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:26.829634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:26.829634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:26.829634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:26.829634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:26.829634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:26.830513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:26.831439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:26.831439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:26.831439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:26.831439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:26.831439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:26.831439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:26.831439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:26.831439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:26.835437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 271/534] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:26.836278 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884ed4e-db7b-4417-a3a1-79c4a085c0ea None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:26.895205 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c28f7-f413-4a58-8776-7cf9c4957762 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:26.931301 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e94e65-1367-4fba-b216-88f4d2156317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:26.931908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e94e65-1367-4fba-b216-88f4d2156317 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:26.932171 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e94e65-1367-4fba-b216-88f4d2156317 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:26.932446 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e94e65-1367-4fba-b216-88f4d2156317 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:26.933090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e94e65-1367-4fba-b216-88f4d2156317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:26.933457 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e94e65-1367-4fba-b216-88f4d2156317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:26.935535 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e94e65-1367-4fba-b216-88f4d2156317 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:26.942015 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884ed4e-db7b-4417-a3a1-79c4a085c0ea None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:26.951355 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 264/535] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:26.952247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e94e65-1367-4fba-b216-88f4d2156317 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:26.964856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:26.978360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadf6cf-b9a6-45fd-98f2-e800802e7806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:26.978983 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadf6cf-b9a6-45fd-98f2-e800802e7806 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:26.979296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadf6cf-b9a6-45fd-98f2-e800802e7806 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:26.979570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadf6cf-b9a6-45fd-98f2-e800802e7806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:26.980885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cadf6cf-b9a6-45fd-98f2-e800802e7806 None admin] RBAC: Adding query filter params (role.id=244ad93ecd1949c9a6e8f74b135b43b4, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=cfda7307e703427faaf599e32b91bc91) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:26.981522 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cadf6cf-b9a6-45fd-98f2-e800802e7806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:26.982873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cadf6cf-b9a6-45fd-98f2-e800802e7806 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:26.989479 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 272/536] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:38:26 2022] GET /identity/v3/role_assignments?user.id=cfda7307e703427faaf599e32b91bc91&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=244ad93ecd1949c9a6e8f74b135b43b4 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:26.990321 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadf6cf-b9a6-45fd-98f2-e800802e7806 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:27.030511 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:27.068361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e94e65-1367-4fba-b216-88f4d2156317 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:27.116274 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ce2b9-7475-4684-992e-75bd0ad5cee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:27.116900 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ce2b9-7475-4684-992e-75bd0ad5cee6 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:27.117189 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ce2b9-7475-4684-992e-75bd0ad5cee6 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:27.117467 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ce2b9-7475-4684-992e-75bd0ad5cee6 None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:27.118187 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ce2b9-7475-4684-992e-75bd0ad5cee6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:27.118492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ce2b9-7475-4684-992e-75bd0ad5cee6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:27.120656 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ce2b9-7475-4684-992e-75bd0ad5cee6 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:27.134664 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 265/537] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:27.141419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ce2b9-7475-4684-992e-75bd0ad5cee6 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:27.172535 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadf6cf-b9a6-45fd-98f2-e800802e7806 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:27.211278 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a2186-e6a4-411b-9a4b-0e0c98d1df73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:27.212255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a2186-e6a4-411b-9a4b-0e0c98d1df73 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:27.212760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a2186-e6a4-411b-9a4b-0e0c98d1df73 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:27.213523 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:27.214425 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a2186-e6a4-411b-9a4b-0e0c98d1df73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:27.221713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a2186-e6a4-411b-9a4b-0e0c98d1df73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:27.223469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a2186-e6a4-411b-9a4b-0e0c98d1df73 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:27.244647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-d83a2186-e6a4-411b-9a4b-0e0c98d1df73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:27.244647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:27.244647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:27.244647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:27.244647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:27.244647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:27.244647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:27.244647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:27.244647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:27.244647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:27.244647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:27.244647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:27.244647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:27.244647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:27.244647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:27.245556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:27.246964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:27.246964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:27.246964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:27.246964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:27.246964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:27.246964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:27.246964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:27.246964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:27.249558 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 273/538] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:27.256908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ce2b9-7475-4684-992e-75bd0ad5cee6 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:27.262242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a2186-e6a4-411b-9a4b-0e0c98d1df73 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:27.321399 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95260f0-456c-47ad-b4f5-0378bc56d064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:27.321927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95260f0-456c-47ad-b4f5-0378bc56d064 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:27.322237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95260f0-456c-47ad-b4f5-0378bc56d064 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:27.322513 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95260f0-456c-47ad-b4f5-0378bc56d064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:27.323369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95260f0-456c-47ad-b4f5-0378bc56d064 None admin] RBAC: Adding query filter params (role.id=244ad93ecd1949c9a6e8f74b135b43b4, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=ef389656cc8546fdb7d9f30329b93dbd) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:27.329163 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95260f0-456c-47ad-b4f5-0378bc56d064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:27.329163 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95260f0-456c-47ad-b4f5-0378bc56d064 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:27.331779 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 266/539] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:38:27 2022] GET /identity/v3/role_assignments?user.id=ef389656cc8546fdb7d9f30329b93dbd&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=244ad93ecd1949c9a6e8f74b135b43b4 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:27.367531 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:27.388689 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a2186-e6a4-411b-9a4b-0e0c98d1df73 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:27.427293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e538d-21ee-400c-b6db-60eb93950388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:27.427902 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e538d-21ee-400c-b6db-60eb93950388 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:27.428184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e538d-21ee-400c-b6db-60eb93950388 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:27.428456 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e538d-21ee-400c-b6db-60eb93950388 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:27.429234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e538d-21ee-400c-b6db-60eb93950388 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:27.429541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e538d-21ee-400c-b6db-60eb93950388 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:27.431709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e538d-21ee-400c-b6db-60eb93950388 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:27.446850 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 274/540] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:27.454033 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95260f0-456c-47ad-b4f5-0378bc56d064 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:27.564944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95260f0-456c-47ad-b4f5-0378bc56d064 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:27.597599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:27.610212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d42a246-b78d-46e0-bc88-acc30595ef49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:27.610821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42a246-b78d-46e0-bc88-acc30595ef49 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:27.611117 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42a246-b78d-46e0-bc88-acc30595ef49 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:27.615824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d42a246-b78d-46e0-bc88-acc30595ef49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:27.615824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d42a246-b78d-46e0-bc88-acc30595ef49 None admin] RBAC: Adding query filter params (role.id=244ad93ecd1949c9a6e8f74b135b43b4, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=7bff0664e0b44b7c85344a9f3e88f126) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:27.615824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d42a246-b78d-46e0-bc88-acc30595ef49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:27.615824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d42a246-b78d-46e0-bc88-acc30595ef49 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:27.620064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 267/541] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:38:27 2022] GET /identity/v3/role_assignments?user.id=7bff0664e0b44b7c85344a9f3e88f126&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=244ad93ecd1949c9a6e8f74b135b43b4 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:28.016325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ac900-7d51-4684-89be-697666efbe54 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:28.017187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ac900-7d51-4684-89be-697666efbe54 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:28.017604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ac900-7d51-4684-89be-697666efbe54 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:28.019312 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 275/542] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:28.024690 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73421393-2b27-40be-96ae-1c80b662b97c None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:28.025480 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73421393-2b27-40be-96ae-1c80b662b97c None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:28.026223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73421393-2b27-40be-96ae-1c80b662b97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:28.044493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-73421393-2b27-40be-96ae-1c80b662b97c None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:28.045978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-73421393-2b27-40be-96ae-1c80b662b97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:28.113647 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 268/543] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:28.487376 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ac900-7d51-4684-89be-697666efbe54 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:28.592482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ac900-7d51-4684-89be-697666efbe54 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:28.626398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12484c-fbbd-4473-bc45-43dcbac88a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:28.627019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12484c-fbbd-4473-bc45-43dcbac88a57 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:28.627329 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12484c-fbbd-4473-bc45-43dcbac88a57 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:28.627733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12484c-fbbd-4473-bc45-43dcbac88a57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:28.630449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:28.640396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12484c-fbbd-4473-bc45-43dcbac88a57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:28.645280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-2a12484c-fbbd-4473-bc45-43dcbac88a57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:28.645280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:28.645280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:28.645280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:28.645280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:28.645280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:28.645280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:28.645280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:28.645280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:28.645280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:28.645280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:28.645280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:28.645280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:28.645280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:28.645280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:28.645280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:28.650795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:28.650795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:28.650795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:28.650795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:28.650795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:28.650795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:28.650795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:28.650795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:28.650795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:28.650795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:28.650795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:28.650795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:28.650795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:28.650795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:28.650795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:28.650795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:28.652765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:28.652765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:28.652765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:28.652765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:28.652765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:28.652765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:28.652765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:28.652765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:28.652765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:28.652765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:28.652765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:28.652765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:28.652765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 276/544] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:28.653427 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73421393-2b27-40be-96ae-1c80b662b97c None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:28.779208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73421393-2b27-40be-96ae-1c80b662b97c None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:28.810276 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:28.823138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d2b10-8f76-46c5-a204-71108b67dcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:28.823773 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d2b10-8f76-46c5-a204-71108b67dcdb None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:28.825577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d2b10-8f76-46c5-a204-71108b67dcdb None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:28.825577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d2b10-8f76-46c5-a204-71108b67dcdb None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:28.825577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d2b10-8f76-46c5-a204-71108b67dcdb None admin] RBAC: Adding query filter params (name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:28.825803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d2b10-8f76-46c5-a204-71108b67dcdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:28.827246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d2b10-8f76-46c5-a204-71108b67dcdb None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:28.838589 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 269/545] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:28.843736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12484c-fbbd-4473-bc45-43dcbac88a57 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:28.972045 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12484c-fbbd-4473-bc45-43dcbac88a57 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:29.017407 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b0d4b-afb7-4262-b4b7-f2fa24750d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:29.018196 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b0d4b-afb7-4262-b4b7-f2fa24750d8e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:29.018320 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b0d4b-afb7-4262-b4b7-f2fa24750d8e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:29.018599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b0d4b-afb7-4262-b4b7-f2fa24750d8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:29.034939 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b0d4b-afb7-4262-b4b7-f2fa24750d8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:29.036355 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b0d4b-afb7-4262-b4b7-f2fa24750d8e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:29.056245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-369b0d4b-afb7-4262-b4b7-f2fa24750d8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:29.056245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:29.056245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:29.056245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:29.056245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:29.056245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:29.056245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:29.056245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:29.056245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:29.056245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:29.056245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:29.056245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:29.056245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:29.056245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:29.056245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:29.056245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:29.057166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:29.058139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:29.058139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:29.058139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:29.058139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 277/546] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:29.068201 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d2b10-8f76-46c5-a204-71108b67dcdb None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:29.080820 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7a424-3309-4b76-846a-5076c33578c8 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:29.081103 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7a424-3309-4b76-846a-5076c33578c8 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:29.081379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7a424-3309-4b76-846a-5076c33578c8 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:29.083002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 278/547] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:38:29.087875 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5eb9b-ccec-46f4-8554-bf672e3ffb3f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:29.088153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5eb9b-ccec-46f4-8554-bf672e3ffb3f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:29.088427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5eb9b-ccec-46f4-8554-bf672e3ffb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:29.107554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-42e5eb9b-ccec-46f4-8554-bf672e3ffb3f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:29.108854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-42e5eb9b-ccec-46f4-8554-bf672e3ffb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:29.173849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:29.227137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 279/548] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:38:29.239411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d2b10-8f76-46c5-a204-71108b67dcdb None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:29.287434 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cc529-d576-42b9-bd84-2cf49bc16166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:29.287968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cc529-d576-42b9-bd84-2cf49bc16166 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:29.288245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cc529-d576-42b9-bd84-2cf49bc16166 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:29.288514 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cc529-d576-42b9-bd84-2cf49bc16166 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:29.289155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cc529-d576-42b9-bd84-2cf49bc16166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:29.289498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cc529-d576-42b9-bd84-2cf49bc16166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:29.290860 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cc529-d576-42b9-bd84-2cf49bc16166 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:29.309351 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 270/549] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:29.314993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5eb9b-ccec-46f4-8554-bf672e3ffb3f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:29.409341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87a959-16ab-48f5-9333-0edd6a5857ed None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:29.409633 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87a959-16ab-48f5-9333-0edd6a5857ed None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:29.409871 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87a959-16ab-48f5-9333-0edd6a5857ed None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:29.411211 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 271/550] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:38:29.425922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fa241-ca86-4c9b-b3b4-67d5c9e2b72e None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:29.426190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fa241-ca86-4c9b-b3b4-67d5c9e2b72e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:29.426453 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fa241-ca86-4c9b-b3b4-67d5c9e2b72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:29.456758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-ef6fa241-ca86-4c9b-b3b4-67d5c9e2b72e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:29.460235 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5eb9b-ccec-46f4-8554-bf672e3ffb3f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:29.461870 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ef6fa241-ca86-4c9b-b3b4-67d5c9e2b72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:29.520707 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210115b-30cb-482c-a195-af0685546c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:29.521338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210115b-30cb-482c-a195-af0685546c32 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:29.521613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210115b-30cb-482c-a195-af0685546c32 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:29.521877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210115b-30cb-482c-a195-af0685546c32 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:29.538473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210115b-30cb-482c-a195-af0685546c32 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:29.539900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210115b-30cb-482c-a195-af0685546c32 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:29.540666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:29.558000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-0210115b-30cb-482c-a195-af0685546c32 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:38:29.558000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:29.558000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:29.558000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:29.558000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:29.558000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:29.558000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:29.558000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:29.558000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:29.558000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:29.558000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:29.558000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:29.558000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:29.558000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:29.558000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:29.558000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:29.561033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:29.563315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:29.563315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:29.563315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:29.563315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:29.563315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:29.563315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:29.563315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:29.563315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:29.563315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:38:29.563315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:29.563315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 280/551] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:38:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:29.571217 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210115b-30cb-482c-a195-af0685546c32 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:29.585883 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 272/552] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:29.638297 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fa241-ca86-4c9b-b3b4-67d5c9e2b72e None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:29.643925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:29.754178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fa241-ca86-4c9b-b3b4-67d5c9e2b72e None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:29.774809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210115b-30cb-482c-a195-af0685546c32 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:29.795040 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e922db-4db5-40a2-ba97-1e00f45f718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:29.795668 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e922db-4db5-40a2-ba97-1e00f45f718c None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:29.795974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e922db-4db5-40a2-ba97-1e00f45f718c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:29.796248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e922db-4db5-40a2-ba97-1e00f45f718c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:29.809490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e922db-4db5-40a2-ba97-1e00f45f718c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:29.813408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5e15b-81b0-44dc-8f22-d6c08e6ea841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:29.814176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-84e922db-4db5-40a2-ba97-1e00f45f718c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:29.814176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:29.814176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:29.814176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:29.814176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:29.814176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:29.814176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:29.814176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:29.814176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:29.814176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:29.814176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:29.814176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:29.814176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:29.814176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:29.814176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:29.814176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:29.815093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:29.815093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:29.815093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:29.815093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:29.815093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:29.815093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:29.815093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:29.815093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:29.815093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:29.815093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:29.815093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:29.815093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:29.815093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:29.815093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:29.815093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:29.815093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:29.815957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:29.815957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:29.815957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:29.815957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:29.815957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:29.815957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:29.815957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:29.815957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:29.815957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:29.815957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:29.815957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:29.815957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:29.815957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 273/553] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:38:29.816754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fccf69-4739-4463-a0f9-d8132bfbe421 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:29.822060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fccf69-4739-4463-a0f9-d8132bfbe421 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:29.822060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fccf69-4739-4463-a0f9-d8132bfbe421 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:29.822060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 274/554] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:29.822323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5e15b-81b0-44dc-8f22-d6c08e6ea841 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:29.822323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5e15b-81b0-44dc-8f22-d6c08e6ea841 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:29.822323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5e15b-81b0-44dc-8f22-d6c08e6ea841 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:29.822323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5e15b-81b0-44dc-8f22-d6c08e6ea841 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:29.822323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5e15b-81b0-44dc-8f22-d6c08e6ea841 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:29.823010 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5e15b-81b0-44dc-8f22-d6c08e6ea841 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:29.824397 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fccf69-4739-4463-a0f9-d8132bfbe421 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:29.844960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 281/555] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Aug 30 14:38:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:29.851173 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2db83-2ae8-42aa-a630-adf82772c501 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:29.851722 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2db83-2ae8-42aa-a630-adf82772c501 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:29.852239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2db83-2ae8-42aa-a630-adf82772c501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:29.866242 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:29.872440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-76a2db83-2ae8-42aa-a630-adf82772c501 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:29.873627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-76a2db83-2ae8-42aa-a630-adf82772c501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:29.938596 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fccf69-4739-4463-a0f9-d8132bfbe421 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:29.945660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 282/556] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:29.946632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2db83-2ae8-42aa-a630-adf82772c501 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:29.977458 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:29.984487 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6f095-fe4b-47b2-a3b3-7f62bcfdd0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:29.986089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6f095-fe4b-47b2-a3b3-7f62bcfdd0f9 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:29.986545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6f095-fe4b-47b2-a3b3-7f62bcfdd0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:29.986986 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6f095-fe4b-47b2-a3b3-7f62bcfdd0f9 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:29.987872 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6f095-fe4b-47b2-a3b3-7f62bcfdd0f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:29.988344 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6f095-fe4b-47b2-a3b3-7f62bcfdd0f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:29.989869 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6f095-fe4b-47b2-a3b3-7f62bcfdd0f9 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:30.006177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 275/557] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:30.008292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6f095-fe4b-47b2-a3b3-7f62bcfdd0f9 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:30.072822 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2db83-2ae8-42aa-a630-adf82772c501 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:30.112209 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e25d6-1ce8-4c07-aabd-e0112283a90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:30.113192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e25d6-1ce8-4c07-aabd-e0112283a90e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:30.113654 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e25d6-1ce8-4c07-aabd-e0112283a90e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:30.114106 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e25d6-1ce8-4c07-aabd-e0112283a90e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:30.141888 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:30.147743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6f095-fe4b-47b2-a3b3-7f62bcfdd0f9 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:30.183705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08524b-924a-49c6-b5d1-dddf4662c6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:30.184327 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08524b-924a-49c6-b5d1-dddf4662c6f3 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:30.184607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08524b-924a-49c6-b5d1-dddf4662c6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:30.184878 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08524b-924a-49c6-b5d1-dddf4662c6f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:30.187709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e25d6-1ce8-4c07-aabd-e0112283a90e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:30.189059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e25d6-1ce8-4c07-aabd-e0112283a90e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:30.196032 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08524b-924a-49c6-b5d1-dddf4662c6f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:30.200877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-9f08524b-924a-49c6-b5d1-dddf4662c6f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:30.200877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:30.200877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:30.200877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:30.200877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:30.200877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:30.200877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:30.200877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:30.200877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:30.200877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:30.200877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:30.200877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:30.200877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:30.200877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:30.200877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:30.200877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:30.201834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:30.201834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:30.201834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:30.201834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:30.201834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:30.201834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:30.201834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:30.201834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:30.201834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:30.201834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:30.201834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:30.201834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:30.201834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:30.201834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:30.201834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:30.201834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:30.202651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:30.202651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:30.202651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:30.202651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:30.202651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:30.202651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:30.202651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:30.202651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:30.202651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:30.202651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:30.202651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:30.202651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:30.202651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 276/558] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:30.203828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08524b-924a-49c6-b5d1-dddf4662c6f3 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:30.223439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:30.300502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 283/559] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:30.301769 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ee977-67e2-4ff2-b7b9-6277f4b96567 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:30.302059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ee977-67e2-4ff2-b7b9-6277f4b96567 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:30.302341 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ee977-67e2-4ff2-b7b9-6277f4b96567 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:30.303757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 284/560] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:30.306151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08524b-924a-49c6-b5d1-dddf4662c6f3 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:30.307867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ee977-67e2-4ff2-b7b9-6277f4b96567 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:30.340189 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf94c8-158c-481e-b2d9-107cd446b989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:30.340766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf94c8-158c-481e-b2d9-107cd446b989 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:30.341047 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf94c8-158c-481e-b2d9-107cd446b989 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:30.341320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf94c8-158c-481e-b2d9-107cd446b989 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:30.351390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf94c8-158c-481e-b2d9-107cd446b989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:30.352830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf94c8-158c-481e-b2d9-107cd446b989 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:30.367424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-76cf94c8-158c-481e-b2d9-107cd446b989 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:30.367424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:30.367424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:30.367424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:30.367424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:30.367424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:30.367424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:30.367424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:30.367424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:30.367424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:30.367424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:30.367424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:30.367424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:30.367424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:30.367424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:30.367424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:30.368395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:30.369251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:30.369251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:30.369251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:30.369251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 277/561] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:30.369251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf94c8-158c-481e-b2d9-107cd446b989 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:30.383543 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:30.413315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ee977-67e2-4ff2-b7b9-6277f4b96567 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:30.438663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:30.446930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ca8dd-9b9d-4c47-a98a-8a0e9a9fdaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:30.447571 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ca8dd-9b9d-4c47-a98a-8a0e9a9fdaa9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:30.447851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ca8dd-9b9d-4c47-a98a-8a0e9a9fdaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:30.448127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ca8dd-9b9d-4c47-a98a-8a0e9a9fdaa9 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:30.448815 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ca8dd-9b9d-4c47-a98a-8a0e9a9fdaa9 None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:30.449800 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ca8dd-9b9d-4c47-a98a-8a0e9a9fdaa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:30.451216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ca8dd-9b9d-4c47-a98a-8a0e9a9fdaa9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:30.459800 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 285/562] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:30.460430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ca8dd-9b9d-4c47-a98a-8a0e9a9fdaa9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:30.472781 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf94c8-158c-481e-b2d9-107cd446b989 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:30.507498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf14633-8a18-4952-90f5-61ff0208f052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:30.508138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf14633-8a18-4952-90f5-61ff0208f052 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:30.508405 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf14633-8a18-4952-90f5-61ff0208f052 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:30.508679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf14633-8a18-4952-90f5-61ff0208f052 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:30.519342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf14633-8a18-4952-90f5-61ff0208f052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:30.524504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-aaf14633-8a18-4952-90f5-61ff0208f052 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:30.524504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:30.524504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:30.524504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:30.524504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:30.524504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:30.524504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:30.524504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:30.524504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:30.524504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:30.524504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:30.524504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:30.524504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:30.524504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:30.524504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:30.524504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:30.525564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:30.525564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:30.525564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:30.525564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:30.525564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:30.525564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:30.525564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:30.525564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:30.525564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:30.525564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:30.525564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:30.525564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:30.525564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:30.525564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:30.525564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:30.525564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:30.527022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:30.527022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:30.527022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:30.527022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:30.527022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:30.527022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:30.527022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:30.527022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:30.527022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:30.527022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:30.527022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:30.527022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:30.527022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 278/563] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:30.527022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771fe72-44a7-4f32-ab29-9641e25b4ef0 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:30.527804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771fe72-44a7-4f32-ab29-9641e25b4ef0 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:30.527804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771fe72-44a7-4f32-ab29-9641e25b4ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:30.545891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-b771fe72-44a7-4f32-ab29-9641e25b4ef0 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:30.547083 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b771fe72-44a7-4f32-ab29-9641e25b4ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:30.565109 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ca8dd-9b9d-4c47-a98a-8a0e9a9fdaa9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:30.598058 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3114002-8836-493a-a2d2-ab1b79f07177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:30.598609 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3114002-8836-493a-a2d2-ab1b79f07177 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:30.598884 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3114002-8836-493a-a2d2-ab1b79f07177 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:30.599180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3114002-8836-493a-a2d2-ab1b79f07177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:30.608055 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:30.609238 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3114002-8836-493a-a2d2-ab1b79f07177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:30.610679 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3114002-8836-493a-a2d2-ab1b79f07177 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:30.613147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 279/564] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:30.613925 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771fe72-44a7-4f32-ab29-9641e25b4ef0 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:30.626622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-e3114002-8836-493a-a2d2-ab1b79f07177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:30.626622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:30.626622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:30.626622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:30.626622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:30.626622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:30.626622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:30.626622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:30.626622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:30.626622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:30.626622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:30.626622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:30.626622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:30.626622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:30.626622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:30.626622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:30.627592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:30.628495 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:30.628495 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:30.628495 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:30.632373 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 286/565] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:30.633208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3114002-8836-493a-a2d2-ab1b79f07177 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:30.677508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:30.723719 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771fe72-44a7-4f32-ab29-9641e25b4ef0 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:30.739854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3114002-8836-493a-a2d2-ab1b79f07177 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:30.762012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3a259-c038-42ea-bbb5-646be47f3737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:30.762613 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3a259-c038-42ea-bbb5-646be47f3737 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:30.762888 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3a259-c038-42ea-bbb5-646be47f3737 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:30.763179 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3a259-c038-42ea-bbb5-646be47f3737 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:30.763814 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3a259-c038-42ea-bbb5-646be47f3737 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:30.764117 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3a259-c038-42ea-bbb5-646be47f3737 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:30.765460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3a259-c038-42ea-bbb5-646be47f3737 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:30.776403 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd75711-dbc0-4043-8998-b0e263dd7058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:30.777072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 280/566] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:30.777476 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd75711-dbc0-4043-8998-b0e263dd7058 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:30.777849 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3a259-c038-42ea-bbb5-646be47f3737 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:30.778117 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd75711-dbc0-4043-8998-b0e263dd7058 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:30.778567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd75711-dbc0-4043-8998-b0e263dd7058 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:30.792091 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd75711-dbc0-4043-8998-b0e263dd7058 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:30.794297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd75711-dbc0-4043-8998-b0e263dd7058 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:30.809505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-4dd75711-dbc0-4043-8998-b0e263dd7058 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:30.809505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:30.809505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:30.809505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:30.809505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:30.809505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:30.809505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:30.809505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:30.809505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:30.809505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:30.809505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:30.809505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:30.809505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:30.809505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:30.809505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:30.809505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:30.810653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:30.811762 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:30.811762 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:30.811762 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:30.811762 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 287/567] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:30.812032 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd75711-dbc0-4043-8998-b0e263dd7058 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:30.844534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:30.890495 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3a259-c038-42ea-bbb5-646be47f3737 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:30.906101 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:30.915513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd75711-dbc0-4043-8998-b0e263dd7058 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:30.923943 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfc9ba-af9d-44c7-a3b3-dd91f93d02f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:30.924555 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfc9ba-af9d-44c7-a3b3-dd91f93d02f3 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:30.924829 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfc9ba-af9d-44c7-a3b3-dd91f93d02f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:30.925112 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfc9ba-af9d-44c7-a3b3-dd91f93d02f3 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:30.925797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfc9ba-af9d-44c7-a3b3-dd91f93d02f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:30.926055 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfc9ba-af9d-44c7-a3b3-dd91f93d02f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:30.927438 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfc9ba-af9d-44c7-a3b3-dd91f93d02f3 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:30.935873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 281/568] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:30.936744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfc9ba-af9d-44c7-a3b3-dd91f93d02f3 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:30.951209 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b422ff4-25c9-4063-ab48-66b33fd00835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:30.951805 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b422ff4-25c9-4063-ab48-66b33fd00835 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:30.952085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b422ff4-25c9-4063-ab48-66b33fd00835 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:30.952353 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b422ff4-25c9-4063-ab48-66b33fd00835 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:30.952989 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b422ff4-25c9-4063-ab48-66b33fd00835 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:30.953290 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b422ff4-25c9-4063-ab48-66b33fd00835 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:30.954644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b422ff4-25c9-4063-ab48-66b33fd00835 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:30.965538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 288/569] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:30.966300 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b422ff4-25c9-4063-ab48-66b33fd00835 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:31.034968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfc9ba-af9d-44c7-a3b3-dd91f93d02f3 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:31.053553 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:31.063115 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b422ff4-25c9-4063-ab48-66b33fd00835 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:31.067260 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698524aa-9fc4-432b-b43b-5300a7dcbe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:31.067875 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698524aa-9fc4-432b-b43b-5300a7dcbe5d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:31.068149 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698524aa-9fc4-432b-b43b-5300a7dcbe5d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:31.068414 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698524aa-9fc4-432b-b43b-5300a7dcbe5d None admin] PATH_INFO: `/v3/users/nova` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:31.076136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698524aa-9fc4-432b-b43b-5300a7dcbe5d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:31.077545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698524aa-9fc4-432b-b43b-5300a7dcbe5d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:31.088836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-698524aa-9fc4-432b-b43b-5300a7dcbe5d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:38:31.088836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:31.088836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:31.088836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:31.088836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:31.088836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:31.088836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:31.088836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:31.088836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:31.088836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:31.088836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:31.088836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:31.088836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:31.088836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:31.088836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:31.088836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:31.089948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:31.090873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:31.090873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:31.090873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:31.090873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:31.090873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:31.090873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:31.090873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:31.090873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:31.090873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:38:31.090873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:31.094556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 282/570] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 30 14:38:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:31.095442 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698524aa-9fc4-432b-b43b-5300a7dcbe5d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:31.096731 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d700f-0625-4e49-8f73-9959274f12a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:31.097356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d700f-0625-4e49-8f73-9959274f12a5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:31.097639 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d700f-0625-4e49-8f73-9959274f12a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:31.097915 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d700f-0625-4e49-8f73-9959274f12a5 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:31.098561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d700f-0625-4e49-8f73-9959274f12a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:31.098869 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d700f-0625-4e49-8f73-9959274f12a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:31.100267 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d700f-0625-4e49-8f73-9959274f12a5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:31.111741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 289/571] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:31.112519 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d700f-0625-4e49-8f73-9959274f12a5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:31.127057 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:31.195534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698524aa-9fc4-432b-b43b-5300a7dcbe5d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:31.210366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d700f-0625-4e49-8f73-9959274f12a5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:31.228688 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb5644-a66a-4751-a210-cf38d9940e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:31.229308 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb5644-a66a-4751-a210-cf38d9940e88 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:31.229586 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb5644-a66a-4751-a210-cf38d9940e88 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:31.229858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb5644-a66a-4751-a210-cf38d9940e88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:31.240142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb5644-a66a-4751-a210-cf38d9940e88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:31.245368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-5ffb5644-a66a-4751-a210-cf38d9940e88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:31.245368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:31.245368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:31.245368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:31.245368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:31.245368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:31.245368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:31.245368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:31.245368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:31.245368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:31.245368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:31.245368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:31.245368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:31.245368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:31.245368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:31.245368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:31.246807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:31.246807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:31.246807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:31.246807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:31.246807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:31.246807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:31.246807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:31.246807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:31.246807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:31.246807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:31.246807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:31.246807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:31.246807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:31.246807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:31.246807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:31.246807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:31.247997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:31.247997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:31.247997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:31.247997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:31.247997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:31.247997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:31.247997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:31.247997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:31.247997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:31.247997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:31.247997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:31.247997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:31.247997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 283/572] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:31.247997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb5644-a66a-4751-a210-cf38d9940e88 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:31.252698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05f291-7a50-490e-a21c-ae2019e1114b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:31.253328 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05f291-7a50-490e-a21c-ae2019e1114b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:31.253607 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05f291-7a50-490e-a21c-ae2019e1114b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:31.253876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05f291-7a50-490e-a21c-ae2019e1114b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:31.266049 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05f291-7a50-490e-a21c-ae2019e1114b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:31.267544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05f291-7a50-490e-a21c-ae2019e1114b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:31.283060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-ad05f291-7a50-490e-a21c-ae2019e1114b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:31.283060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:31.283060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:31.283060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:31.283060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:31.283060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:31.283060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:31.283060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:31.283060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:31.283060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:31.283060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:31.283060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:31.283060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:31.283060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:31.283060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:31.283060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:31.284193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:31.285340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:31.285340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:31.285340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:31.285340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 290/573] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:31.285340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05f291-7a50-490e-a21c-ae2019e1114b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:31.294074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:31.345872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:31.350817 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb5644-a66a-4751-a210-cf38d9940e88 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:31.383065 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05f291-7a50-490e-a21c-ae2019e1114b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:31.383944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240bedca-0d2d-4128-8327-9439dd52beb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:31.384556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bedca-0d2d-4128-8327-9439dd52beb5 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:31.384826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bedca-0d2d-4128-8327-9439dd52beb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:31.385095 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bedca-0d2d-4128-8327-9439dd52beb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:31.391527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240bedca-0d2d-4128-8327-9439dd52beb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:31.392938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240bedca-0d2d-4128-8327-9439dd52beb5 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:31.404461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-240bedca-0d2d-4128-8327-9439dd52beb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:31.404461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:31.404461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:31.404461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:31.404461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:31.404461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:31.404461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:31.404461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:31.404461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:31.404461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:31.404461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:31.404461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:31.404461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:31.404461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:31.404461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:31.405511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:31.406579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:31.406579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:31.406579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:31.406579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:31.406579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:31.406579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:31.406579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:31.406579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:31.406579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 284/574] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:31.406579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240bedca-0d2d-4128-8327-9439dd52beb5 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:31.419249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e6f17-b0c7-421f-b031-c3ed10fbd6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:31.419851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e6f17-b0c7-421f-b031-c3ed10fbd6e5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:31.420130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e6f17-b0c7-421f-b031-c3ed10fbd6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:31.420402 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e6f17-b0c7-421f-b031-c3ed10fbd6e5 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:31.421133 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e6f17-b0c7-421f-b031-c3ed10fbd6e5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:31.421441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e6f17-b0c7-421f-b031-c3ed10fbd6e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:31.422885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e6f17-b0c7-421f-b031-c3ed10fbd6e5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:31.440056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 291/575] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 30 14:38:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:31.440835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e6f17-b0c7-421f-b031-c3ed10fbd6e5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:31.505268 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:31.508094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240bedca-0d2d-4128-8327-9439dd52beb5 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:31.537628 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e6f17-b0c7-421f-b031-c3ed10fbd6e5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:31.541858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb638e-6759-4eb7-811b-d4aa1610d24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:31.542971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb638e-6759-4eb7-811b-d4aa1610d24d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:31.543294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb638e-6759-4eb7-811b-d4aa1610d24d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:31.543568 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb638e-6759-4eb7-811b-d4aa1610d24d None admin] PATH_INFO: `/v3/users/glance` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:31.550408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb638e-6759-4eb7-811b-d4aa1610d24d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:31.551853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb638e-6759-4eb7-811b-d4aa1610d24d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:31.560470 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:31.563912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-dcbb638e-6759-4eb7-811b-d4aa1610d24d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:38:31.563912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:31.563912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:31.563912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:31.563912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:31.563912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:31.563912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:31.563912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:31.563912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:31.563912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:31.563912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:31.563912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:31.563912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:31.563912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:31.563912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:31.563912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:31.583074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:31.584479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:31.584479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:31.584479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:31.584479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:31.584479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:31.584479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:31.584479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:31.584479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:31.584479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:38:31.584479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:31.584479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 285/576] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:31.584479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb638e-6759-4eb7-811b-d4aa1610d24d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:31.585328 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec643fc-a3b0-4bac-a2ca-67b80d2aaf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:31.585328 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec643fc-a3b0-4bac-a2ca-67b80d2aaf82 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:31.585328 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec643fc-a3b0-4bac-a2ca-67b80d2aaf82 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:31.585328 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec643fc-a3b0-4bac-a2ca-67b80d2aaf82 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:31.585634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec643fc-a3b0-4bac-a2ca-67b80d2aaf82 None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:31.585634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec643fc-a3b0-4bac-a2ca-67b80d2aaf82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:31.585634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec643fc-a3b0-4bac-a2ca-67b80d2aaf82 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:31.588898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 292/577] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:31.589663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec643fc-a3b0-4bac-a2ca-67b80d2aaf82 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:31.666864 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb638e-6759-4eb7-811b-d4aa1610d24d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:31.686837 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec643fc-a3b0-4bac-a2ca-67b80d2aaf82 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:31.701151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1ab65-a301-4e7d-90e3-12613a478d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:31.701742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1ab65-a301-4e7d-90e3-12613a478d85 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:31.702023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1ab65-a301-4e7d-90e3-12613a478d85 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:31.702296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1ab65-a301-4e7d-90e3-12613a478d85 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:31.702937 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e1ab65-a301-4e7d-90e3-12613a478d85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:31.703285 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e1ab65-a301-4e7d-90e3-12613a478d85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:31.704654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e1ab65-a301-4e7d-90e3-12613a478d85 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:31.716075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 286/578] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:31.716848 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1ab65-a301-4e7d-90e3-12613a478d85 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:31.722069 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1e78f-c136-4943-9595-e24ec0d350b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:31.722670 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1e78f-c136-4943-9595-e24ec0d350b0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:31.722976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1e78f-c136-4943-9595-e24ec0d350b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:31.723284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1e78f-c136-4943-9595-e24ec0d350b0 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:31.723981 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1e78f-c136-4943-9595-e24ec0d350b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:31.724290 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1e78f-c136-4943-9595-e24ec0d350b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:31.725714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1e78f-c136-4943-9595-e24ec0d350b0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:31.737472 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 293/579] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:31.738257 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1e78f-c136-4943-9595-e24ec0d350b0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:31.741912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:31.785054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:31.821581 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1ab65-a301-4e7d-90e3-12613a478d85 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:31.839536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1e78f-c136-4943-9595-e24ec0d350b0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:31.854804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e4933-7ecb-4374-b4b5-5d70a667ef6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:31.855445 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e4933-7ecb-4374-b4b5-5d70a667ef6e None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:31.855716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e4933-7ecb-4374-b4b5-5d70a667ef6e None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:31.855986 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e4933-7ecb-4374-b4b5-5d70a667ef6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:31.865649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e4933-7ecb-4374-b4b5-5d70a667ef6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:31.867060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e4933-7ecb-4374-b4b5-5d70a667ef6e None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:31.873290 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d479674-b069-4a92-ba75-d193c82a603e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:31.873882 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d479674-b069-4a92-ba75-d193c82a603e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:31.874158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d479674-b069-4a92-ba75-d193c82a603e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:31.874422 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d479674-b069-4a92-ba75-d193c82a603e None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:31.875164 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d479674-b069-4a92-ba75-d193c82a603e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:31.875472 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d479674-b069-4a92-ba75-d193c82a603e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:31.876882 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d479674-b069-4a92-ba75-d193c82a603e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:31.881766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-2a9e4933-7ecb-4374-b4b5-5d70a667ef6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:31.881766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:31.881766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:31.881766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:31.881766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:31.881766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:31.881766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:31.881766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:31.881766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:31.881766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:31.881766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:31.881766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:31.881766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:31.881766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:31.881766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:31.881766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:31.882816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:31.883827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:31.883827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:31.883827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:31.883827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 287/580] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:31.883827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e4933-7ecb-4374-b4b5-5d70a667ef6e None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:31.898779 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 294/581] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Aug 30 14:38:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:31.899710 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d479674-b069-4a92-ba75-d193c82a603e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:31.951875 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:31.979968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e4933-7ecb-4374-b4b5-5d70a667ef6e None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:31.999744 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d479674-b069-4a92-ba75-d193c82a603e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:32.001346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:32.013330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62c8b1-8fe4-4914-8cf1-e7ce000c0984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:32.013947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62c8b1-8fe4-4914-8cf1-e7ce000c0984 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:32.014222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62c8b1-8fe4-4914-8cf1-e7ce000c0984 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:32.014490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62c8b1-8fe4-4914-8cf1-e7ce000c0984 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:32.024889 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62c8b1-8fe4-4914-8cf1-e7ce000c0984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:32.026578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62c8b1-8fe4-4914-8cf1-e7ce000c0984 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:32.032673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5277f32-3c72-4591-a44b-82adfa12e2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:32.033280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5277f32-3c72-4591-a44b-82adfa12e2e7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:32.033580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5277f32-3c72-4591-a44b-82adfa12e2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:32.033844 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5277f32-3c72-4591-a44b-82adfa12e2e7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:32.040245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-fb62c8b1-8fe4-4914-8cf1-e7ce000c0984 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:32.040245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:32.040245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:32.040245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:32.040245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:32.040245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:32.040245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:32.040245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:32.040245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:32.040245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:32.040245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:32.040245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:32.040245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:32.040245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:32.040245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:32.040245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:32.041282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:32.042173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:32.042173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:32.042173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:32.042425 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5277f32-3c72-4591-a44b-82adfa12e2e7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:32.043024 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 288/582] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:32.043122 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5277f32-3c72-4591-a44b-82adfa12e2e7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:32.043195 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62c8b1-8fe4-4914-8cf1-e7ce000c0984 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:32.055590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-a5277f32-3c72-4591-a44b-82adfa12e2e7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:38:32.055590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:32.055590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:32.055590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:32.055590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:32.055590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:32.055590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:32.055590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:32.055590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:32.055590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:32.055590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:32.055590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:32.055590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:32.055590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:32.055590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:32.055590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:32.056742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:32.057700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:32.057700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:32.057700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:32.057700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:32.057700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:32.057700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:32.057700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:32.057700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:32.057700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:38:32.057700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:32.057700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 295/583] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:32.058850 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5277f32-3c72-4591-a44b-82adfa12e2e7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:32.143529 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62c8b1-8fe4-4914-8cf1-e7ce000c0984 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:32.153539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5277f32-3c72-4591-a44b-82adfa12e2e7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:32.166295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:32.177050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55012a70-0596-49b9-8ce8-49d590e132a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:32.177641 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55012a70-0596-49b9-8ce8-49d590e132a1 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:32.177915 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55012a70-0596-49b9-8ce8-49d590e132a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:32.178187 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55012a70-0596-49b9-8ce8-49d590e132a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:32.178896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55012a70-0596-49b9-8ce8-49d590e132a1 None admin] RBAC: Adding query filter params (role.id=244ad93ecd1949c9a6e8f74b135b43b4, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=bc23480d5aa94cad9e6599762ce01548) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:32.179232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55012a70-0596-49b9-8ce8-49d590e132a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:32.180645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55012a70-0596-49b9-8ce8-49d590e132a1 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:32.186266 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 289/584] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:38:32 2022] GET /identity/v3/role_assignments?user.id=bc23480d5aa94cad9e6599762ce01548&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=244ad93ecd1949c9a6e8f74b135b43b4 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:32.187019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55012a70-0596-49b9-8ce8-49d590e132a1 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:32.187145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faecad-8808-45df-9f66-1ca5493cf94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:32.187791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faecad-8808-45df-9f66-1ca5493cf94d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:32.188126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faecad-8808-45df-9f66-1ca5493cf94d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:32.188455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faecad-8808-45df-9f66-1ca5493cf94d None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:32.189169 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55faecad-8808-45df-9f66-1ca5493cf94d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:32.189540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55faecad-8808-45df-9f66-1ca5493cf94d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:32.190931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55faecad-8808-45df-9f66-1ca5493cf94d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:32.204441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 296/585] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:32.209817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faecad-8808-45df-9f66-1ca5493cf94d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:32.241535 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:32.302704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55012a70-0596-49b9-8ce8-49d590e132a1 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:32.349491 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faecad-8808-45df-9f66-1ca5493cf94d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:32.354449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578d491-333e-4d06-bc54-93273e93b074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:32.355608 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578d491-333e-4d06-bc54-93273e93b074 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:32.355891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578d491-333e-4d06-bc54-93273e93b074 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:32.356165 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578d491-333e-4d06-bc54-93273e93b074 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:32.367239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578d491-333e-4d06-bc54-93273e93b074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:32.368681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578d491-333e-4d06-bc54-93273e93b074 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:32.383585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-f578d491-333e-4d06-bc54-93273e93b074 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:32.383585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:32.383585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:32.383585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:32.383585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:32.383585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:32.383585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:32.383585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:32.383585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:32.383585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:32.383585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:32.383585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:32.383585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:32.383585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:32.383585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:32.383585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:32.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:32.385396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:32.385396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:32.385396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:32.388017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8a4ac-dbe2-425a-aae1-649709105993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:32.388624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8a4ac-dbe2-425a-aae1-649709105993 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:32.388903 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8a4ac-dbe2-425a-aae1-649709105993 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:32.389432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 290/586] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:32.389544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8a4ac-dbe2-425a-aae1-649709105993 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:32.390034 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578d491-333e-4d06-bc54-93273e93b074 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:32.390129 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8a4ac-dbe2-425a-aae1-649709105993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:32.390335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8a4ac-dbe2-425a-aae1-649709105993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:32.391742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8a4ac-dbe2-425a-aae1-649709105993 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:32.405450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 297/587] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:32.406337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8a4ac-dbe2-425a-aae1-649709105993 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:32.441056 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:32.495468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:32.503702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578d491-333e-4d06-bc54-93273e93b074 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:32.517878 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8a4ac-dbe2-425a-aae1-649709105993 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:32.539695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e475df-6759-41a0-af4f-dcde21a5148f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:32.540289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e475df-6759-41a0-af4f-dcde21a5148f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:32.540563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e475df-6759-41a0-af4f-dcde21a5148f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:32.540832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e475df-6759-41a0-af4f-dcde21a5148f None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:32.541549 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e475df-6759-41a0-af4f-dcde21a5148f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:32.541850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e475df-6759-41a0-af4f-dcde21a5148f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:32.543817 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e475df-6759-41a0-af4f-dcde21a5148f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:32.558890 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f14bff-8465-419e-bd5b-6c97153ff1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:32.559497 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f14bff-8465-419e-bd5b-6c97153ff1c9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:32.559778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f14bff-8465-419e-bd5b-6c97153ff1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:32.560058 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f14bff-8465-419e-bd5b-6c97153ff1c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:32.560708 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 291/588] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Aug 30 14:38:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:32.561519 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e475df-6759-41a0-af4f-dcde21a5148f None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:32.571952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f14bff-8465-419e-bd5b-6c97153ff1c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:32.573547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f14bff-8465-419e-bd5b-6c97153ff1c9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:32.585741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-e6f14bff-8465-419e-bd5b-6c97153ff1c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:32.585741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:32.585741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:32.585741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:32.585741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:32.585741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:32.585741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:32.585741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:32.585741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:32.585741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:32.585741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:32.585741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:32.585741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:32.585741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:32.585741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:32.586618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:32.588227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:32.588227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:32.588227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:32.588227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:32.588227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:32.588227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:32.588227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:32.588227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:32.588227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 298/589] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:32.588227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f14bff-8465-419e-bd5b-6c97153ff1c9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:32.673446 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:32.678183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e475df-6759-41a0-af4f-dcde21a5148f None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:32.694557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f14bff-8465-419e-bd5b-6c97153ff1c9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:32.718314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22161a-5c81-4a6a-9bb3-426af09728ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:32.719378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22161a-5c81-4a6a-9bb3-426af09728ab None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:32.719939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22161a-5c81-4a6a-9bb3-426af09728ab None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:32.720224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22161a-5c81-4a6a-9bb3-426af09728ab None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:32.721143 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22161a-5c81-4a6a-9bb3-426af09728ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:32.721454 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22161a-5c81-4a6a-9bb3-426af09728ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:32.723521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22161a-5c81-4a6a-9bb3-426af09728ab None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:32.729295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52483c-3285-4bc2-820c-3cac97d6aa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:32.729927 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52483c-3285-4bc2-820c-3cac97d6aa56 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:32.730724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52483c-3285-4bc2-820c-3cac97d6aa56 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:32.730988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52483c-3285-4bc2-820c-3cac97d6aa56 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:32.741451 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 292/590] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:32.741451 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22161a-5c81-4a6a-9bb3-426af09728ab None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:32.741674 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52483c-3285-4bc2-820c-3cac97d6aa56 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:32.741674 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52483c-3285-4bc2-820c-3cac97d6aa56 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:32.751249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-aa52483c-3285-4bc2-820c-3cac97d6aa56 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:38:32.751249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:32.751249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:32.751249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:32.751249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:32.751249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:32.751249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:32.751249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:32.751249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:32.751249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:32.751249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:32.751249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:32.751249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:32.751249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:32.751249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:32.751249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:32.752384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:32.753796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:32.753796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:32.753796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:32.753796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:32.753796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:32.753796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:32.753796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:32.753796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:32.753796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:38:32.753796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:32.753796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 299/591] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:32.756396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52483c-3285-4bc2-820c-3cac97d6aa56 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:32.759137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:32.858998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22161a-5c81-4a6a-9bb3-426af09728ab None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:32.861464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52483c-3285-4bc2-820c-3cac97d6aa56 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:32.893989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7f9bf-2ebd-41d3-ac9d-9aebeb2e1ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:32.894704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7f9bf-2ebd-41d3-ac9d-9aebeb2e1ef3 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:32.895020 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7f9bf-2ebd-41d3-ac9d-9aebeb2e1ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:32.895388 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7f9bf-2ebd-41d3-ac9d-9aebeb2e1ef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:32.895483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0be98-b978-4f85-9e74-49c16c51ef6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:32.896152 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0be98-b978-4f85-9e74-49c16c51ef6f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:32.896503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0be98-b978-4f85-9e74-49c16c51ef6f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:32.896844 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0be98-b978-4f85-9e74-49c16c51ef6f None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:32.897610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0be98-b978-4f85-9e74-49c16c51ef6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:32.897987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0be98-b978-4f85-9e74-49c16c51ef6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:32.899540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0be98-b978-4f85-9e74-49c16c51ef6f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:32.907123 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7f9bf-2ebd-41d3-ac9d-9aebeb2e1ef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:32.908381 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7f9bf-2ebd-41d3-ac9d-9aebeb2e1ef3 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:32.916724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 300/592] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:32.916724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0be98-b978-4f85-9e74-49c16c51ef6f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:32.927005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-a3e7f9bf-2ebd-41d3-ac9d-9aebeb2e1ef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:32.927005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:32.927005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:32.927005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:32.927005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:32.927005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:32.927005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:32.927005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:32.927005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:32.927005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:32.927005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:32.927005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:32.927005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:32.927005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:32.927005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:32.927005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:32.928145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:32.929195 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:32.929195 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:32.929195 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:32.931422 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 293/593] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:32.932628 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7f9bf-2ebd-41d3-ac9d-9aebeb2e1ef3 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:32.946387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:32.983388 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:33.023703 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0be98-b978-4f85-9e74-49c16c51ef6f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:33.036928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7f9bf-2ebd-41d3-ac9d-9aebeb2e1ef3 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:33.062547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430b433-6a63-4e6b-8034-1ec82197ab1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:33.064094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430b433-6a63-4e6b-8034-1ec82197ab1f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:33.064725 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430b433-6a63-4e6b-8034-1ec82197ab1f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:33.065363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430b433-6a63-4e6b-8034-1ec82197ab1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:33.073563 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430b433-6a63-4e6b-8034-1ec82197ab1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:33.074572 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff1389-2d23-4a08-b2ee-192db4fe0c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:33.075190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff1389-2d23-4a08-b2ee-192db4fe0c26 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:33.075436 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430b433-6a63-4e6b-8034-1ec82197ab1f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:33.075599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff1389-2d23-4a08-b2ee-192db4fe0c26 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:33.075779 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff1389-2d23-4a08-b2ee-192db4fe0c26 None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:33.076442 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff1389-2d23-4a08-b2ee-192db4fe0c26 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:33.076740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff1389-2d23-4a08-b2ee-192db4fe0c26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:33.078380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff1389-2d23-4a08-b2ee-192db4fe0c26 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:33.088179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-7430b433-6a63-4e6b-8034-1ec82197ab1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:33.088179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:33.088179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:33.088179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:33.088179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:33.088179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:33.088179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:33.088179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:33.088179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:33.088179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:33.088179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:33.088179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:33.088179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:33.088179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:33.088179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:33.089244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:33.090344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:33.090344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:33.090344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:33.090344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:33.090344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:33.090344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:33.090344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:33.090344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:33.091656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 301/594] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:33.091998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430b433-6a63-4e6b-8034-1ec82197ab1f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:33.108508 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 294/595] 10.222.0.27 () {56 vars in 1139 bytes} [Tue Aug 30 14:38:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:33.109250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff1389-2d23-4a08-b2ee-192db4fe0c26 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:33.175675 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:33.197328 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430b433-6a63-4e6b-8034-1ec82197ab1f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:33.210593 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff1389-2d23-4a08-b2ee-192db4fe0c26 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:33.221503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:33.235366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0686d8-4aa4-40b9-af73-8cf921d1e171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:33.236029 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0686d8-4aa4-40b9-af73-8cf921d1e171 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:33.236311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0686d8-4aa4-40b9-af73-8cf921d1e171 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:33.236651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0686d8-4aa4-40b9-af73-8cf921d1e171 None admin] PATH_INFO: `/v3/projects/1147574d59c7404baf76a24149684f6e/users/5b6e4c5bf2a54b28a55e5ae74b90ab4f/roles/244ad93ecd1949c9a6e8f74b135b43b4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:33.251057 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5f9ca-0be9-4379-ab52-c25bd02113a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:33.251704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5f9ca-0be9-4379-ab52-c25bd02113a5 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:33.251981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5f9ca-0be9-4379-ab52-c25bd02113a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:33.252250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5f9ca-0be9-4379-ab52-c25bd02113a5 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:33.252910 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5f9ca-0be9-4379-ab52-c25bd02113a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:33.253208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5f9ca-0be9-4379-ab52-c25bd02113a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:33.254584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5f9ca-0be9-4379-ab52-c25bd02113a5 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:33.255416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0686d8-4aa4-40b9-af73-8cf921d1e171 None admin] RBAC: Authorizing `identity:create_grant(project_id=1147574d59c7404baf76a24149684f6e, user_id=5b6e4c5bf2a54b28a55e5ae74b90ab4f, role_id=244ad93ecd1949c9a6e8f74b135b43b4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:33.257081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0686d8-4aa4-40b9-af73-8cf921d1e171 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:33.268194 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 295/596] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:33.269017 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5f9ca-0be9-4379-ab52-c25bd02113a5 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:33.323432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 302/597] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:38:33 2022] PUT /identity/v3/projects/1147574d59c7404baf76a24149684f6e/users/5b6e4c5bf2a54b28a55e5ae74b90ab4f/roles/244ad93ecd1949c9a6e8f74b135b43b4 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:38:33.324583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0686d8-4aa4-40b9-af73-8cf921d1e171 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:33.391853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5f9ca-0be9-4379-ab52-c25bd02113a5 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:33.428731 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:33.437425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38231e27-ee74-457f-8b40-e98be7150813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:33.438074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38231e27-ee74-457f-8b40-e98be7150813 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:33.445303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38231e27-ee74-457f-8b40-e98be7150813 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:33.445303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38231e27-ee74-457f-8b40-e98be7150813 None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:33.445303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38231e27-ee74-457f-8b40-e98be7150813 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:33.445303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38231e27-ee74-457f-8b40-e98be7150813 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:33.445303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38231e27-ee74-457f-8b40-e98be7150813 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:33.454407 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 296/598] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:33.455489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38231e27-ee74-457f-8b40-e98be7150813 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:33.479008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0686d8-4aa4-40b9-af73-8cf921d1e171 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:33.520048 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962a66d-99cf-4a8c-9b7f-183c7453fab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:33.520838 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962a66d-99cf-4a8c-9b7f-183c7453fab2 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:33.521233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962a66d-99cf-4a8c-9b7f-183c7453fab2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:33.521605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962a66d-99cf-4a8c-9b7f-183c7453fab2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:33.535738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962a66d-99cf-4a8c-9b7f-183c7453fab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:33.537332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962a66d-99cf-4a8c-9b7f-183c7453fab2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:33.556575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-a962a66d-99cf-4a8c-9b7f-183c7453fab2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:33.556575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:33.556575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:33.556575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:33.556575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:33.556575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:33.556575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:33.556575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:33.556575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:33.556575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:33.556575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:33.556575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:33.556575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:33.556575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:33.556575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:33.556575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:33.557740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:33.558988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:33.558988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:33.558988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:33.558988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 303/599] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:33.560271 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962a66d-99cf-4a8c-9b7f-183c7453fab2 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:33.580145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38231e27-ee74-457f-8b40-e98be7150813 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:33.602594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:33.617915 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a66f8b-52d2-47ab-8610-f3ddfd7e7d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:33.618676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a66f8b-52d2-47ab-8610-f3ddfd7e7d2e None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:33.619089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a66f8b-52d2-47ab-8610-f3ddfd7e7d2e None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:33.619481 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a66f8b-52d2-47ab-8610-f3ddfd7e7d2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:33.628125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a66f8b-52d2-47ab-8610-f3ddfd7e7d2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:33.629642 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a66f8b-52d2-47ab-8610-f3ddfd7e7d2e None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:33.642884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-83a66f8b-52d2-47ab-8610-f3ddfd7e7d2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:33.642884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:33.642884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:33.642884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:33.642884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:33.642884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:33.642884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:33.642884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:33.642884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:33.642884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:33.642884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:33.642884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:33.642884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:33.642884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:33.642884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:33.643972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:33.645048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:33.645048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:33.645048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:33.645048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:33.645048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:33.645048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:33.645048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:33.645048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:33.652583 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 297/600] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:33.653341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a66f8b-52d2-47ab-8610-f3ddfd7e7d2e None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:33.690133 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962a66d-99cf-4a8c-9b7f-183c7453fab2 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:33.707920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:33.742672 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f39ed-ccab-49fa-b04a-b34292643fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:33.744058 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f39ed-ccab-49fa-b04a-b34292643fab None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:33.744799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f39ed-ccab-49fa-b04a-b34292643fab None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:33.745221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f39ed-ccab-49fa-b04a-b34292643fab None admin] PATH_INFO: `/v3/projects/1147574d59c7404baf76a24149684f6e/users/cfda7307e703427faaf599e32b91bc91/roles/244ad93ecd1949c9a6e8f74b135b43b4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:33.766581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f39ed-ccab-49fa-b04a-b34292643fab None admin] RBAC: Authorizing `identity:create_grant(project_id=1147574d59c7404baf76a24149684f6e, user_id=cfda7307e703427faaf599e32b91bc91, role_id=244ad93ecd1949c9a6e8f74b135b43b4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:33.767131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a66f8b-52d2-47ab-8610-f3ddfd7e7d2e None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:33.768813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f39ed-ccab-49fa-b04a-b34292643fab None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:33.802774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449689ea-8faa-4da1-a2d3-3d2413797c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:33.803573 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449689ea-8faa-4da1-a2d3-3d2413797c09 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:33.803950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449689ea-8faa-4da1-a2d3-3d2413797c09 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:33.804320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449689ea-8faa-4da1-a2d3-3d2413797c09 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:33.805069 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449689ea-8faa-4da1-a2d3-3d2413797c09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:33.805468 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449689ea-8faa-4da1-a2d3-3d2413797c09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:33.806943 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449689ea-8faa-4da1-a2d3-3d2413797c09 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:33.819384 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 298/601] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:33.820351 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449689ea-8faa-4da1-a2d3-3d2413797c09 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:33.864194 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 304/602] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:38:33 2022] PUT /identity/v3/projects/1147574d59c7404baf76a24149684f6e/users/cfda7307e703427faaf599e32b91bc91/roles/244ad93ecd1949c9a6e8f74b135b43b4 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:38:33.865377 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f39ed-ccab-49fa-b04a-b34292643fab None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:33.991705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449689ea-8faa-4da1-a2d3-3d2413797c09 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:34.010185 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:34.019889 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:34.056304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7cd24-59d6-43d3-9bad-4618e045c7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:34.057380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7cd24-59d6-43d3-9bad-4618e045c7a3 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:34.057380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7cd24-59d6-43d3-9bad-4618e045c7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:34.057848 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7cd24-59d6-43d3-9bad-4618e045c7a3 None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:34.059198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7cd24-59d6-43d3-9bad-4618e045c7a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:34.059605 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7cd24-59d6-43d3-9bad-4618e045c7a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:34.061148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7cd24-59d6-43d3-9bad-4618e045c7a3 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:34.061567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f39ed-ccab-49fa-b04a-b34292643fab None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:34.080558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 299/603] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:34.081865 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7cd24-59d6-43d3-9bad-4618e045c7a3 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:34.107352 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf8fb82-b054-4468-baeb-1f4fd11e708b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:34.108242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8fb82-b054-4468-baeb-1f4fd11e708b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:34.108518 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8fb82-b054-4468-baeb-1f4fd11e708b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:34.108933 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8fb82-b054-4468-baeb-1f4fd11e708b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:34.117503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf8fb82-b054-4468-baeb-1f4fd11e708b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:34.119247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf8fb82-b054-4468-baeb-1f4fd11e708b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:34.131364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-1bf8fb82-b054-4468-baeb-1f4fd11e708b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:34.131364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:34.131364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:34.131364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:34.131364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:34.131364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:34.131364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:34.131364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:34.131364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:34.131364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:34.131364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:34.131364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:34.131364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:34.131364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:34.131364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:34.132455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:34.133467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:34.133467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:34.133467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:34.133467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:34.133467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:34.133467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:34.133467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:34.133467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:34.135151 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 305/604] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:34.137017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841dd89-b176-4a2f-be21-ffa64003821d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:34.137437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841dd89-b176-4a2f-be21-ffa64003821d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:34.137848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841dd89-b176-4a2f-be21-ffa64003821d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:34.142804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 306/605] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:34.142804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841dd89-b176-4a2f-be21-ffa64003821d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:34.201431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7cd24-59d6-43d3-9bad-4618e045c7a3 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:34.240834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a63c1-88bc-4156-9709-a1001b9c6572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:34.241875 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a63c1-88bc-4156-9709-a1001b9c6572 None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:34.242293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a63c1-88bc-4156-9709-a1001b9c6572 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:34.242840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a63c1-88bc-4156-9709-a1001b9c6572 None admin] PATH_INFO: `/v3/projects/1147574d59c7404baf76a24149684f6e/users/ef389656cc8546fdb7d9f30329b93dbd/roles/244ad93ecd1949c9a6e8f74b135b43b4` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:34.259679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a63c1-88bc-4156-9709-a1001b9c6572 None admin] RBAC: Authorizing `identity:create_grant(project_id=1147574d59c7404baf76a24149684f6e, user_id=ef389656cc8546fdb7d9f30329b93dbd, role_id=244ad93ecd1949c9a6e8f74b135b43b4)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:34.263867 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a63c1-88bc-4156-9709-a1001b9c6572 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:34.266132 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841dd89-b176-4a2f-be21-ffa64003821d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:34.303608 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d229ae-79a9-4636-b292-2395537ea9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:34.304220 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d229ae-79a9-4636-b292-2395537ea9a1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:34.304498 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d229ae-79a9-4636-b292-2395537ea9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:34.304768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d229ae-79a9-4636-b292-2395537ea9a1 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:34.305456 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d229ae-79a9-4636-b292-2395537ea9a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:34.305760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d229ae-79a9-4636-b292-2395537ea9a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:34.307242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d229ae-79a9-4636-b292-2395537ea9a1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:34.309557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:34.320432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 307/606] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:34.326109 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a9d06-96a8-4fe1-9496-c6f47f715b28 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:34.326513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a9d06-96a8-4fe1-9496-c6f47f715b28 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:34.326985 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a9d06-96a8-4fe1-9496-c6f47f715b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:34.345708 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-fc3a9d06-96a8-4fe1-9496-c6f47f715b28 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:34.346898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fc3a9d06-96a8-4fe1-9496-c6f47f715b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:34.388934 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 300/607] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:38:34 2022] PUT /identity/v3/projects/1147574d59c7404baf76a24149684f6e/users/ef389656cc8546fdb7d9f30329b93dbd/roles/244ad93ecd1949c9a6e8f74b135b43b4 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:38:34.390147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a63c1-88bc-4156-9709-a1001b9c6572 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:34.435466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:34.458638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 308/608] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:34.563336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a63c1-88bc-4156-9709-a1001b9c6572 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:34.597663 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddad61-9219-4730-82cc-acb3fb5d4d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:34.598316 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddad61-9219-4730-82cc-acb3fb5d4d7f None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:34.598592 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddad61-9219-4730-82cc-acb3fb5d4d7f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:34.598856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddad61-9219-4730-82cc-acb3fb5d4d7f None admin] PATH_INFO: `/v3/projects/1147574d59c7404baf76a24149684f6e/users/7bff0664e0b44b7c85344a9f3e88f126/roles/244ad93ecd1949c9a6e8f74b135b43b4` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:34.613755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddad61-9219-4730-82cc-acb3fb5d4d7f None admin] RBAC: Authorizing `identity:create_grant(project_id=1147574d59c7404baf76a24149684f6e, user_id=7bff0664e0b44b7c85344a9f3e88f126, role_id=244ad93ecd1949c9a6e8f74b135b43b4)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:34.615197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddad61-9219-4730-82cc-acb3fb5d4d7f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:34.684454 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 301/609] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:38:34 2022] PUT /identity/v3/projects/1147574d59c7404baf76a24149684f6e/users/7bff0664e0b44b7c85344a9f3e88f126/roles/244ad93ecd1949c9a6e8f74b135b43b4 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:38:34.811214 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a9d06-96a8-4fe1-9496-c6f47f715b28 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:34.928419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a9d06-96a8-4fe1-9496-c6f47f715b28 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:34.952609 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:34.968168 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e74751-3d57-4dfb-b07a-825b9ce610fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:34.968861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e74751-3d57-4dfb-b07a-825b9ce610fd None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:34.969214 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e74751-3d57-4dfb-b07a-825b9ce610fd None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:34.971775 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e74751-3d57-4dfb-b07a-825b9ce610fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:34.982188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e74751-3d57-4dfb-b07a-825b9ce610fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:34.987265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-a3e74751-3d57-4dfb-b07a-825b9ce610fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:34.987265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:34.987265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:34.987265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:34.987265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:34.987265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:34.987265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:34.987265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:34.987265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:34.987265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:34.987265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:34.987265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:34.987265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:34.987265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:34.987265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:34.987265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:34.988189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:34.988189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:34.988189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:34.988189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:34.988189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:34.988189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:34.988189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:34.988189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:34.988189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:34.988189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:34.988189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:34.988189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:34.988189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:34.988189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:34.988189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:34.988189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:34.989205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:34.989205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:34.989205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:34.989205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:34.989205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:34.989205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:34.989205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:34.989205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:34.989205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:34.989205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:34.989205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:34.989205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:34.989205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 309/610] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:34.997270 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddad61-9219-4730-82cc-acb3fb5d4d7f None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:35.080358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:35.134043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddad61-9219-4730-82cc-acb3fb5d4d7f None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:35.192876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b54d18-4268-4e06-b085-98997cd96094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:35.193404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b54d18-4268-4e06-b085-98997cd96094 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:35.193684 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b54d18-4268-4e06-b085-98997cd96094 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:35.193955 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b54d18-4268-4e06-b085-98997cd96094 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:35.194624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b54d18-4268-4e06-b085-98997cd96094 None admin] RBAC: Adding query filter params (name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:35.194932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b54d18-4268-4e06-b085-98997cd96094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:35.196321 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b54d18-4268-4e06-b085-98997cd96094 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:35.211153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 302/611] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:35.217789 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e74751-3d57-4dfb-b07a-825b9ce610fd None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:35.368880 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e74751-3d57-4dfb-b07a-825b9ce610fd None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:35.411871 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5adaf-b857-472d-8f63-277fe89e1494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:35.412500 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5adaf-b857-472d-8f63-277fe89e1494 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:35.412776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5adaf-b857-472d-8f63-277fe89e1494 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:35.413044 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5adaf-b857-472d-8f63-277fe89e1494 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:35.425249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5adaf-b857-472d-8f63-277fe89e1494 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:35.426735 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5adaf-b857-472d-8f63-277fe89e1494 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:35.450561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-a6b5adaf-b857-472d-8f63-277fe89e1494 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:35.450561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:35.450561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:35.450561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:35.450561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:35.450561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:35.450561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:35.450561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:35.450561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:35.450561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:35.450561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:35.450561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:35.450561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:35.450561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:35.450561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:35.450561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:35.451515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:35.452580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:35.452580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:35.452580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:35.452580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 310/612] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:35.458515 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b54d18-4268-4e06-b085-98997cd96094 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:35.499655 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565d4a6-1939-4186-b0bb-eb899232e44c None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:35.499951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565d4a6-1939-4186-b0bb-eb899232e44c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:35.500202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565d4a6-1939-4186-b0bb-eb899232e44c None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:35.501787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 311/613] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:35.507580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e32cb-74cc-4ec9-8f59-7304696ade25 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:35.507867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e32cb-74cc-4ec9-8f59-7304696ade25 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:35.508135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e32cb-74cc-4ec9-8f59-7304696ade25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:35.530627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-dd8e32cb-74cc-4ec9-8f59-7304696ade25 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:35.532274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-dd8e32cb-74cc-4ec9-8f59-7304696ade25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:35.567146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:35.626171 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 312/614] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:38:35.656647 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b54d18-4268-4e06-b085-98997cd96094 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:35.702403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:35.708434 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e1454-dae3-4210-bfed-ed66957dace7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:35.709037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e1454-dae3-4210-bfed-ed66957dace7 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:35.709572 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e1454-dae3-4210-bfed-ed66957dace7 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:35.709572 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e1454-dae3-4210-bfed-ed66957dace7 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:35.710198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e1454-dae3-4210-bfed-ed66957dace7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:35.710498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e1454-dae3-4210-bfed-ed66957dace7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:35.711895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e1454-dae3-4210-bfed-ed66957dace7 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:35.730818 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 303/615] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:35.736692 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e32cb-74cc-4ec9-8f59-7304696ade25 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:35.895687 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e32cb-74cc-4ec9-8f59-7304696ade25 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:35.923817 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e1454-dae3-4210-bfed-ed66957dace7 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:35.951834 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81340b0-34e8-4913-b352-fc4ef136da84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:35.952549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81340b0-34e8-4913-b352-fc4ef136da84 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:35.952775 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81340b0-34e8-4913-b352-fc4ef136da84 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:35.953052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81340b0-34e8-4913-b352-fc4ef136da84 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:35.964059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81340b0-34e8-4913-b352-fc4ef136da84 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:35.966173 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81340b0-34e8-4913-b352-fc4ef136da84 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:35.992578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-e81340b0-34e8-4913-b352-fc4ef136da84 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:38:35.992578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:35.992578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:35.992578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:35.992578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:35.992578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:35.992578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:35.992578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:35.992578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:35.992578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:35.992578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:35.992578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:35.992578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:35.992578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:35.992578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:35.992578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:35.993593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:35.994550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:35.994550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:35.994550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:35.994550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:35.994550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:35.994550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:35.994550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:35.994550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:35.994550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:38:35.994550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:35.994550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 313/616] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:38:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:35.999987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81340b0-34e8-4913-b352-fc4ef136da84 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:36.031717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e1454-dae3-4210-bfed-ed66957dace7 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:36.051797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:36.066890 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d63c7-9a2c-4c4c-8062-e393a799546c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:36.069819 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d63c7-9a2c-4c4c-8062-e393a799546c None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:36.069819 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d63c7-9a2c-4c4c-8062-e393a799546c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:36.069819 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d63c7-9a2c-4c4c-8062-e393a799546c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:36.080932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d63c7-9a2c-4c4c-8062-e393a799546c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:36.085863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-8f8d63c7-9a2c-4c4c-8062-e393a799546c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:36.085863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:36.085863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:36.085863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:36.085863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:36.085863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:36.085863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:36.085863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:36.085863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:36.085863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:36.085863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:36.085863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:36.085863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:36.085863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:36.085863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:36.085863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:36.086778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:36.086778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:36.086778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:36.086778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:36.086778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:36.086778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:36.086778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:36.086778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:36.086778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:36.086778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:36.086778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:36.086778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:36.086778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:36.086778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:36.086778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:36.086778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:36.087652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:36.087652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:36.087652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:36.087652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:36.087652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:36.087652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:36.087652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:36.087652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:36.087652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:36.087652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:36.087652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:36.087652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:36.087652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 304/617] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:38:36.093558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d63c7-9a2c-4c4c-8062-e393a799546c None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:36.163078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81340b0-34e8-4913-b352-fc4ef136da84 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:36.183692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:36.223271 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0ad41-030b-4365-9021-74f1804998cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:36.223931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0ad41-030b-4365-9021-74f1804998cb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:36.224207 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0ad41-030b-4365-9021-74f1804998cb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:36.224471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0ad41-030b-4365-9021-74f1804998cb None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:36.225198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0ad41-030b-4365-9021-74f1804998cb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:36.225757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0ad41-030b-4365-9021-74f1804998cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:36.227200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0ad41-030b-4365-9021-74f1804998cb None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:36.264286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 314/618] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Aug 30 14:38:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:36.266615 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d63c7-9a2c-4c4c-8062-e393a799546c None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:36.276619 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0ad41-030b-4365-9021-74f1804998cb None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:36.303066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b0f11-c149-48b9-89de-4fe7ffd97f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:36.303884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b0f11-c149-48b9-89de-4fe7ffd97f87 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:36.304025 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b0f11-c149-48b9-89de-4fe7ffd97f87 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:36.304232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b0f11-c149-48b9-89de-4fe7ffd97f87 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:36.304893 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b0f11-c149-48b9-89de-4fe7ffd97f87 None admin] RBAC: Adding query filter params (name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:36.311224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b0f11-c149-48b9-89de-4fe7ffd97f87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:36.312704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b0f11-c149-48b9-89de-4fe7ffd97f87 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:36.326407 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 305/619] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:36.332504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b0f11-c149-48b9-89de-4fe7ffd97f87 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:36.381080 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:36.398545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0ad41-030b-4365-9021-74f1804998cb None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:36.436255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626b24d-2504-4246-b806-a0db439ea8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:36.437254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b24d-2504-4246-b806-a0db439ea8e9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:36.437742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b24d-2504-4246-b806-a0db439ea8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:36.438213 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b24d-2504-4246-b806-a0db439ea8e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:36.441446 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b0f11-c149-48b9-89de-4fe7ffd97f87 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:36.449462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626b24d-2504-4246-b806-a0db439ea8e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:36.451234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626b24d-2504-4246-b806-a0db439ea8e9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:36.463698 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:36.468095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-e626b24d-2504-4246-b806-a0db439ea8e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:36.468095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:36.468095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:36.468095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:36.468095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:36.468095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:36.468095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:36.468095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:36.468095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:36.468095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:36.468095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:36.468095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:36.468095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:36.468095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:36.468095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:36.468095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:36.469033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:36.469945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:36.469945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:36.469945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:36.472028 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 315/620] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:38:36.473467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc489b0-bc00-48d9-b1e2-5a78eb1539de None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:36.473947 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc489b0-bc00-48d9-b1e2-5a78eb1539de None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:36.474417 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc489b0-bc00-48d9-b1e2-5a78eb1539de None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:36.476205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 316/621] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:36.477133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83790f0-e222-47bc-9aae-3ef4c1bd5128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:36.477762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83790f0-e222-47bc-9aae-3ef4c1bd5128 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:36.477995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc489b0-bc00-48d9-b1e2-5a78eb1539de None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:36.483206 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83790f0-e222-47bc-9aae-3ef4c1bd5128 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:36.483206 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83790f0-e222-47bc-9aae-3ef4c1bd5128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:36.490289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83790f0-e222-47bc-9aae-3ef4c1bd5128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:36.491812 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83790f0-e222-47bc-9aae-3ef4c1bd5128 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:36.506607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-f83790f0-e222-47bc-9aae-3ef4c1bd5128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:36.506607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:36.506607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:36.506607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:36.506607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:36.506607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:36.506607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:36.506607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:36.506607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:36.506607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:36.506607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:36.506607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:36.506607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:36.506607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:36.506607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:36.506607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:36.507610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:36.508496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:36.508496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:36.508496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:36.508496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 306/622] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:36.514033 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c204301-544b-4166-9757-f798749e3d25 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:36.514316 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c204301-544b-4166-9757-f798749e3d25 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:36.514590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c204301-544b-4166-9757-f798749e3d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:36.534450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-6c204301-544b-4166-9757-f798749e3d25 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:36.535855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6c204301-544b-4166-9757-f798749e3d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:36.592171 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc489b0-bc00-48d9-b1e2-5a78eb1539de None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:36.605445 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 307/623] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:36.606366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c204301-544b-4166-9757-f798749e3d25 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:36.635813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaf3d3-0f58-496e-9b78-632f1cc2415d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:36.637303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaf3d3-0f58-496e-9b78-632f1cc2415d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:36.637781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaf3d3-0f58-496e-9b78-632f1cc2415d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:36.638249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaf3d3-0f58-496e-9b78-632f1cc2415d None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:36.639127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaf3d3-0f58-496e-9b78-632f1cc2415d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:36.639649 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaf3d3-0f58-496e-9b78-632f1cc2415d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:36.641229 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaf3d3-0f58-496e-9b78-632f1cc2415d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:36.643969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:36.667467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 317/624] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:36.668864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaf3d3-0f58-496e-9b78-632f1cc2415d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:36.711128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:36.716056 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c204301-544b-4166-9757-f798749e3d25 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:36.754082 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a9316-c84e-46e3-81e9-6c2401076ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:36.754693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a9316-c84e-46e3-81e9-6c2401076ed4 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:36.755387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a9316-c84e-46e3-81e9-6c2401076ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:36.755387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a9316-c84e-46e3-81e9-6c2401076ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:36.756015 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a9316-c84e-46e3-81e9-6c2401076ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:36.756321 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a9316-c84e-46e3-81e9-6c2401076ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:36.757727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a9316-c84e-46e3-81e9-6c2401076ed4 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:36.770742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 308/625] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:36.776342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a9316-c84e-46e3-81e9-6c2401076ed4 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:36.884380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a9316-c84e-46e3-81e9-6c2401076ed4 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:36.884573 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaf3d3-0f58-496e-9b78-632f1cc2415d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:36.923525 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cae0ac-4237-4f95-b00e-806c51f4cb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:36.924251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cae0ac-4237-4f95-b00e-806c51f4cb53 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:36.924528 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cae0ac-4237-4f95-b00e-806c51f4cb53 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:36.924798 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cae0ac-4237-4f95-b00e-806c51f4cb53 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:36.927368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8f3c7-1693-45ec-b48c-9e7a7d51dec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:36.928777 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8f3c7-1693-45ec-b48c-9e7a7d51dec5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:36.929056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8f3c7-1693-45ec-b48c-9e7a7d51dec5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:36.929327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8f3c7-1693-45ec-b48c-9e7a7d51dec5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:36.934239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cae0ac-4237-4f95-b00e-806c51f4cb53 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:36.935704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cae0ac-4237-4f95-b00e-806c51f4cb53 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:36.937164 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8f3c7-1693-45ec-b48c-9e7a7d51dec5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:36.939460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8f3c7-1693-45ec-b48c-9e7a7d51dec5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:36.947859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-b1cae0ac-4237-4f95-b00e-806c51f4cb53 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:38:36.947859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:36.947859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:36.947859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:36.947859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:36.947859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:36.947859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:36.947859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:36.947859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:36.947859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:36.947859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:36.947859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:36.947859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:36.947859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:36.947859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:36.947859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:36.949360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:36.950324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:36.950324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:36.950324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:36.950324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:36.950324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:36.950324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:36.950324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:36.950324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:36.950324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:38:36.950324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:36.950324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 309/626] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 30 14:38:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:38:36.955987 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945f29b-2608-44d1-97e6-2ff3178a46a8 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:36.956433 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945f29b-2608-44d1-97e6-2ff3178a46a8 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:36.956783 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945f29b-2608-44d1-97e6-2ff3178a46a8 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:36.958023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-4ba8f3c7-1693-45ec-b48c-9e7a7d51dec5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:36.958023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:36.958023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:36.958023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:36.958023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:36.958023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:36.958023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:36.958023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:36.958023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:36.958023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:36.958023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:36.958023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:36.958023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:36.958023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:36.958023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:36.959754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:36.961031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:36.961031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:36.961031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:36.961031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:36.961031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:36.961031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:36.961031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:36.961031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:36.961716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 310/627] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:36.961716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945f29b-2608-44d1-97e6-2ff3178a46a8 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:36.967361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 318/628] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:36.967361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064834b-a67d-42cd-8d27-c66c1e9507fd None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:36.967361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064834b-a67d-42cd-8d27-c66c1e9507fd None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:36.971324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064834b-a67d-42cd-8d27-c66c1e9507fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:36.980110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:37.000756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b064834b-a67d-42cd-8d27-c66c1e9507fd None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:37.001963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b064834b-a67d-42cd-8d27-c66c1e9507fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:37.052516 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:37.069258 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 319/629] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:37.070330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064834b-a67d-42cd-8d27-c66c1e9507fd None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:37.076780 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945f29b-2608-44d1-97e6-2ff3178a46a8 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:37.111538 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd7aa0-df2d-42f6-9edc-a46e6531db07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:37.112132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd7aa0-df2d-42f6-9edc-a46e6531db07 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:37.112410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd7aa0-df2d-42f6-9edc-a46e6531db07 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:37.112697 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd7aa0-df2d-42f6-9edc-a46e6531db07 None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:37.113412 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd7aa0-df2d-42f6-9edc-a46e6531db07 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:37.113716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd7aa0-df2d-42f6-9edc-a46e6531db07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:37.115203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd7aa0-df2d-42f6-9edc-a46e6531db07 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:37.132979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 311/630] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 30 14:38:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:37.134040 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd7aa0-df2d-42f6-9edc-a46e6531db07 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:37.183786 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064834b-a67d-42cd-8d27-c66c1e9507fd None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:37.212944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:37.218127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b16386-b004-4781-9b07-835c74801bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:37.218737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b16386-b004-4781-9b07-835c74801bfc None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:37.219017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b16386-b004-4781-9b07-835c74801bfc None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:37.219325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b16386-b004-4781-9b07-835c74801bfc None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:37.220056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b16386-b004-4781-9b07-835c74801bfc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:37.220368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b16386-b004-4781-9b07-835c74801bfc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:37.227124 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b16386-b004-4781-9b07-835c74801bfc None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:37.238600 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 320/631] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:37.244470 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411372e2-d4f6-44af-9c51-18b32605b1ae None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:37.244755 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411372e2-d4f6-44af-9c51-18b32605b1ae None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:37.245028 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411372e2-d4f6-44af-9c51-18b32605b1ae None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:37.246852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 321/632] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:37.247593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411372e2-d4f6-44af-9c51-18b32605b1ae None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:37.256867 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd7aa0-df2d-42f6-9edc-a46e6531db07 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:37.298480 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:37.299165 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7e89c-4148-4608-899b-b47901914fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:37.299891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7e89c-4148-4608-899b-b47901914fe5 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:37.300170 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7e89c-4148-4608-899b-b47901914fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:37.300445 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7e89c-4148-4608-899b-b47901914fe5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:37.312352 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7e89c-4148-4608-899b-b47901914fe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:37.317693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-3ac7e89c-4148-4608-899b-b47901914fe5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:37.317693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:37.317693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:37.317693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:37.317693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:37.317693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:37.317693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:37.317693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:37.317693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:37.317693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:37.317693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:37.317693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:37.317693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:37.317693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:37.317693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:37.317693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:37.318620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:37.318620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:37.318620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:37.318620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:37.318620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:37.318620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:37.318620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:37.318620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:37.318620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:37.318620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:37.318620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:37.318620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:37.318620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:37.318620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:37.318620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:37.318620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:37.319489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:37.319489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:37.319489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:37.319489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:37.319489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:37.319489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:37.319489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:37.319489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:37.319489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:37.319489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:37.319489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:37.319489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:37.319489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 312/633] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:37.320199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7e89c-4148-4608-899b-b47901914fe5 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:37.360238 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411372e2-d4f6-44af-9c51-18b32605b1ae None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:37.392929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a7885-293b-4b8e-a89e-cf3d61f7ef1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:37.393534 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a7885-293b-4b8e-a89e-cf3d61f7ef1d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:37.393814 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a7885-293b-4b8e-a89e-cf3d61f7ef1d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:37.394090 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a7885-293b-4b8e-a89e-cf3d61f7ef1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:37.418622 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7e89c-4148-4608-899b-b47901914fe5 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:37.446062 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:37.451374 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ee0a2-919a-4800-9901-41e473dcd6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:37.452020 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ee0a2-919a-4800-9901-41e473dcd6dd None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:37.452299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ee0a2-919a-4800-9901-41e473dcd6dd None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:37.452569 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ee0a2-919a-4800-9901-41e473dcd6dd None admin] PATH_INFO: `/v3/projects/1147574d59c7404baf76a24149684f6e/users/bc23480d5aa94cad9e6599762ce01548/roles/244ad93ecd1949c9a6e8f74b135b43b4` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:37.460249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a7885-293b-4b8e-a89e-cf3d61f7ef1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:37.461706 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a7885-293b-4b8e-a89e-cf3d61f7ef1d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:37.466981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ee0a2-919a-4800-9901-41e473dcd6dd None admin] RBAC: Authorizing `identity:create_grant(project_id=1147574d59c7404baf76a24149684f6e, user_id=bc23480d5aa94cad9e6599762ce01548, role_id=244ad93ecd1949c9a6e8f74b135b43b4)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:37.468432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ee0a2-919a-4800-9901-41e473dcd6dd None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:37.505276 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:37.551190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 313/634] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:38:37 2022] PUT /identity/v3/projects/1147574d59c7404baf76a24149684f6e/users/bc23480d5aa94cad9e6599762ce01548/roles/244ad93ecd1949c9a6e8f74b135b43b4 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:38:37.552386 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de0701-fe7b-44dd-b476-091a350a537b None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:37.552672 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de0701-fe7b-44dd-b476-091a350a537b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:37.552997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de0701-fe7b-44dd-b476-091a350a537b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:37.562591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 322/635] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:37.562591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a7885-293b-4b8e-a89e-cf3d61f7ef1d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:37.581701 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-a9de0701-fe7b-44dd-b476-091a350a537b None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:37.583004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a9de0701-fe7b-44dd-b476-091a350a537b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:37.668729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a7885-293b-4b8e-a89e-cf3d61f7ef1d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:37.672569 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 314/636] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:37.673731 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de0701-fe7b-44dd-b476-091a350a537b None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:37.721917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c1013-4304-48bb-a09e-4148d73bb1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:37.723363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c1013-4304-48bb-a09e-4148d73bb1f4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:37.723363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c1013-4304-48bb-a09e-4148d73bb1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:37.723363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c1013-4304-48bb-a09e-4148d73bb1f4 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:37.723754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c1013-4304-48bb-a09e-4148d73bb1f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:37.724057 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c1013-4304-48bb-a09e-4148d73bb1f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:37.725445 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c1013-4304-48bb-a09e-4148d73bb1f4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:37.735691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 323/637] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:37.736843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c1013-4304-48bb-a09e-4148d73bb1f4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:37.751199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:37.818233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de0701-fe7b-44dd-b476-091a350a537b None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:37.835002 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:37.856823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d828b-7ff2-4579-bfd7-85fa1d3c7ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:37.857384 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d828b-7ff2-4579-bfd7-85fa1d3c7ec8 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:37.857667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d828b-7ff2-4579-bfd7-85fa1d3c7ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:37.857938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d828b-7ff2-4579-bfd7-85fa1d3c7ec8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:37.863432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c1013-4304-48bb-a09e-4148d73bb1f4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:37.869936 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d828b-7ff2-4579-bfd7-85fa1d3c7ec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:37.875527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-990d828b-7ff2-4579-bfd7-85fa1d3c7ec8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:37.875527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:37.875527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:37.875527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:37.875527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:37.875527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:37.875527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:37.875527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:37.875527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:37.875527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:37.875527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:37.875527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:37.875527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:37.875527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:37.875527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:37.875527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:37.877118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:37.877118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:37.877118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:37.877118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:37.877118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:37.877118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:37.877118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:37.877118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:37.877118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:37.877118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:37.877118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:37.877118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:37.877118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:37.877118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:37.877118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:37.877118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:37.878147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:37.878147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:37.878147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:37.878147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:37.878147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:37.878147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:37.878147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:37.878147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:37.878147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:37.878147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:37.878147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:37.878147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:37.881640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 315/638] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:37.882522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d828b-7ff2-4579-bfd7-85fa1d3c7ec8 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:37.911167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced8a81-c522-4fbc-84ad-71387b7950ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:37.912160 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced8a81-c522-4fbc-84ad-71387b7950ec None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:37.912506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced8a81-c522-4fbc-84ad-71387b7950ec None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:37.913097 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced8a81-c522-4fbc-84ad-71387b7950ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:37.927754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced8a81-c522-4fbc-84ad-71387b7950ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:37.930208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced8a81-c522-4fbc-84ad-71387b7950ec None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:37.950665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-fced8a81-c522-4fbc-84ad-71387b7950ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:37.950665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:37.950665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:37.950665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:37.950665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:37.950665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:37.950665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:37.950665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:37.950665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:37.950665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:37.950665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:37.950665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:37.950665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:37.950665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:37.950665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:37.950665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:37.951843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:37.953462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:37.953462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:37.953462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:37.953462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 324/639] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:37.953462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced8a81-c522-4fbc-84ad-71387b7950ec None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:37.993587 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d828b-7ff2-4579-bfd7-85fa1d3c7ec8 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:38.015254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:38.033868 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49285a4-d61e-4c47-9314-46199aff0208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:38.034505 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49285a4-d61e-4c47-9314-46199aff0208 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:38.034785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49285a4-d61e-4c47-9314-46199aff0208 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:38.035057 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49285a4-d61e-4c47-9314-46199aff0208 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:38.045921 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49285a4-d61e-4c47-9314-46199aff0208 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:38.047489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49285a4-d61e-4c47-9314-46199aff0208 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:38.059420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced8a81-c522-4fbc-84ad-71387b7950ec None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:38.062577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-b49285a4-d61e-4c47-9314-46199aff0208 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:38.062577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:38.062577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:38.062577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:38.062577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:38.062577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:38.062577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:38.062577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:38.062577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:38.062577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:38.062577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:38.062577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:38.062577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:38.062577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:38.062577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:38.062577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:38.063536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:38.064409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:38.064409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:38.064409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:38.064409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 316/640] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:38.065128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49285a4-d61e-4c47-9314-46199aff0208 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:38.090964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:38.100883 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f7e21-20a2-466d-a41c-5b98381a98d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:38.101744 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f7e21-20a2-466d-a41c-5b98381a98d2 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:38.102026 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f7e21-20a2-466d-a41c-5b98381a98d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:38.105378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f7e21-20a2-466d-a41c-5b98381a98d2 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:38.105378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f7e21-20a2-466d-a41c-5b98381a98d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:38.105378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f7e21-20a2-466d-a41c-5b98381a98d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:38.105378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f7e21-20a2-466d-a41c-5b98381a98d2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:38.113085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 325/641] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:38.113703 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f7e21-20a2-466d-a41c-5b98381a98d2 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:38.177825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49285a4-d61e-4c47-9314-46199aff0208 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:38.215835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f7e21-20a2-466d-a41c-5b98381a98d2 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:38.220266 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf9974-986d-4044-847c-2735331400ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:38.220894 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf9974-986d-4044-847c-2735331400ae None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:38.221174 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf9974-986d-4044-847c-2735331400ae None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:38.221443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf9974-986d-4044-847c-2735331400ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:38.235335 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf9974-986d-4044-847c-2735331400ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:38.240116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-5bbf9974-986d-4044-847c-2735331400ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:38.240116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:38.240116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:38.240116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:38.240116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:38.240116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:38.240116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:38.240116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:38.240116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:38.240116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:38.240116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:38.240116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:38.240116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:38.240116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:38.240116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:38.240116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:38.241101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:38.241101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:38.241101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:38.241101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:38.241101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:38.241101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:38.241101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:38.241101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:38.241101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:38.241101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:38.241101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:38.241101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:38.241101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:38.241101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:38.241101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:38.241101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:38.241948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:38.241948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:38.241948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:38.241948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:38.241948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:38.241948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:38.241948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:38.241948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:38.241948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:38.241948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:38.241948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:38.241948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:38.241948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 317/642] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:38.242657 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf9974-986d-4044-847c-2735331400ae None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:38.243291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:38.257274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca9f77-c937-4631-aecc-af76e6a724d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:38.257879 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca9f77-c937-4631-aecc-af76e6a724d0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:38.258455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca9f77-c937-4631-aecc-af76e6a724d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:38.258455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca9f77-c937-4631-aecc-af76e6a724d0 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:38.259064 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca9f77-c937-4631-aecc-af76e6a724d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:38.259494 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca9f77-c937-4631-aecc-af76e6a724d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:38.263799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca9f77-c937-4631-aecc-af76e6a724d0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:38.273615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 326/643] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:38.277167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca9f77-c937-4631-aecc-af76e6a724d0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:38.345660 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:38.361101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf9974-986d-4044-847c-2735331400ae None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:38.384316 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca9f77-c937-4631-aecc-af76e6a724d0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:38.396436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94e398-de4f-48d7-a203-1984fc04dd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:38.397032 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94e398-de4f-48d7-a203-1984fc04dd66 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:38.397311 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94e398-de4f-48d7-a203-1984fc04dd66 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:38.397579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94e398-de4f-48d7-a203-1984fc04dd66 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:38.398216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94e398-de4f-48d7-a203-1984fc04dd66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:38.398518 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94e398-de4f-48d7-a203-1984fc04dd66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:38.399915 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94e398-de4f-48d7-a203-1984fc04dd66 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:38.413151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 318/644] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:38.414204 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94e398-de4f-48d7-a203-1984fc04dd66 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:38.422161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f5d8c-538e-46a5-8841-68f4a6e6da16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:38.422790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f5d8c-538e-46a5-8841-68f4a6e6da16 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:38.423067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f5d8c-538e-46a5-8841-68f4a6e6da16 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:38.423372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f5d8c-538e-46a5-8841-68f4a6e6da16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:38.434062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f5d8c-538e-46a5-8841-68f4a6e6da16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:38.435596 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f5d8c-538e-46a5-8841-68f4a6e6da16 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:38.450590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-725f5d8c-538e-46a5-8841-68f4a6e6da16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:38.450590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:38.450590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:38.450590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:38.450590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:38.450590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:38.450590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:38.450590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:38.450590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:38.450590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:38.450590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:38.450590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:38.450590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:38.450590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:38.450590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:38.450590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:38.451544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:38.452601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:38.452601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:38.452601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:38.452601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 327/645] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:38.452601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f5d8c-538e-46a5-8841-68f4a6e6da16 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:38.499731 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:38.532406 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94e398-de4f-48d7-a203-1984fc04dd66 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:38.570987 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85f3e5-dbf0-4c62-8b92-7baeeeb24a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:38.571791 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85f3e5-dbf0-4c62-8b92-7baeeeb24a94 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:38.572075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85f3e5-dbf0-4c62-8b92-7baeeeb24a94 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:38.572349 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85f3e5-dbf0-4c62-8b92-7baeeeb24a94 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:38.573017 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85f3e5-dbf0-4c62-8b92-7baeeeb24a94 None admin] RBAC: Adding query filter params (name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:38.573306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85f3e5-dbf0-4c62-8b92-7baeeeb24a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:38.574696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85f3e5-dbf0-4c62-8b92-7baeeeb24a94 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:38.575212 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f5d8c-538e-46a5-8841-68f4a6e6da16 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:38.583009 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 319/646] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:38.583852 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85f3e5-dbf0-4c62-8b92-7baeeeb24a94 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:38.593825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:38.616413 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49071ae1-5176-4ad9-9c3e-25f9369d97a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:38.616969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49071ae1-5176-4ad9-9c3e-25f9369d97a5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:38.617250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49071ae1-5176-4ad9-9c3e-25f9369d97a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:38.617521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49071ae1-5176-4ad9-9c3e-25f9369d97a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:38.625022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49071ae1-5176-4ad9-9c3e-25f9369d97a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:38.626486 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49071ae1-5176-4ad9-9c3e-25f9369d97a5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:38.638240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-49071ae1-5176-4ad9-9c3e-25f9369d97a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:38.638240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:38.638240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:38.638240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:38.638240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:38.638240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:38.638240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:38.638240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:38.638240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:38.638240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:38.638240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:38.638240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:38.638240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:38.638240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:38.638240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:38.639138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:38.640052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:38.640052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:38.640052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:38.640052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:38.640052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:38.640052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:38.640052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:38.640052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:38.640052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 328/647] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:38.640534 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49071ae1-5176-4ad9-9c3e-25f9369d97a5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:38.696284 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85f3e5-dbf0-4c62-8b92-7baeeeb24a94 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:38.733180 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000bad4-dd27-465c-aee5-a938464fa7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:38.733814 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000bad4-dd27-465c-aee5-a938464fa7ce None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:38.734098 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000bad4-dd27-465c-aee5-a938464fa7ce None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:38.734375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000bad4-dd27-465c-aee5-a938464fa7ce None admin] PATH_INFO: `/v3/users/glance` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:38.742036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000bad4-dd27-465c-aee5-a938464fa7ce None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:38.742913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:38.743841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000bad4-dd27-465c-aee5-a938464fa7ce None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:38.750142 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49071ae1-5176-4ad9-9c3e-25f9369d97a5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:38.755039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-5000bad4-dd27-465c-aee5-a938464fa7ce None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:38:38.755039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:38.755039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:38.755039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:38.755039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:38.755039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:38.755039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:38.755039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:38.755039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:38.755039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:38.755039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:38.755039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:38.755039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:38.755039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:38.755039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:38.755039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:38.756075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:38.756998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:38.756998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:38.756998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:38.756998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:38.756998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:38.756998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:38.756998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:38.756998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:38.756998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:38:38.756998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:38.759399 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 320/648] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:38.760777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000bad4-dd27-465c-aee5-a938464fa7ce None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:38.785378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd0678-de8f-4dad-a564-ebb846fe9b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:38.785990 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd0678-de8f-4dad-a564-ebb846fe9b46 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:38.786281 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd0678-de8f-4dad-a564-ebb846fe9b46 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:38.786540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd0678-de8f-4dad-a564-ebb846fe9b46 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:38.787222 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd0678-de8f-4dad-a564-ebb846fe9b46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:38.787532 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd0678-de8f-4dad-a564-ebb846fe9b46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:38.788950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd0678-de8f-4dad-a564-ebb846fe9b46 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:38.801412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 329/649] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:38.802426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd0678-de8f-4dad-a564-ebb846fe9b46 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:38.833787 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:38.869523 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000bad4-dd27-465c-aee5-a938464fa7ce None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:38.905831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0195f7-951c-4f8c-b6f5-d80b5e61f5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:38.906458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0195f7-951c-4f8c-b6f5-d80b5e61f5b9 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:38.906739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0195f7-951c-4f8c-b6f5-d80b5e61f5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:38.907012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0195f7-951c-4f8c-b6f5-d80b5e61f5b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:38.912998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd0678-de8f-4dad-a564-ebb846fe9b46 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:38.917948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0195f7-951c-4f8c-b6f5-d80b5e61f5b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:38.919425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0195f7-951c-4f8c-b6f5-d80b5e61f5b9 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:38.934137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-6c0195f7-951c-4f8c-b6f5-d80b5e61f5b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:38.934137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:38.934137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:38.934137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:38.934137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:38.934137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:38.934137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:38.934137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:38.934137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:38.934137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:38.934137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:38.934137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:38.934137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:38.934137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:38.934137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:38.934137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:38.935281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:38.936188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:38.936188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:38.936188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:38.936188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 321/650] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:38.936188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0195f7-951c-4f8c-b6f5-d80b5e61f5b9 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:38.953917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e6150-54ab-4c32-b160-e0c11f5d9721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:38.954520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e6150-54ab-4c32-b160-e0c11f5d9721 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:38.954794 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e6150-54ab-4c32-b160-e0c11f5d9721 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:38.955061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e6150-54ab-4c32-b160-e0c11f5d9721 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:38.955779 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e6150-54ab-4c32-b160-e0c11f5d9721 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:38.956085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e6150-54ab-4c32-b160-e0c11f5d9721 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:38.957502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e6150-54ab-4c32-b160-e0c11f5d9721 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:38.975009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 330/651] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:38.975009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e6150-54ab-4c32-b160-e0c11f5d9721 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:38.988407 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:39.055259 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0195f7-951c-4f8c-b6f5-d80b5e61f5b9 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:39.074617 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:39.076585 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e6150-54ab-4c32-b160-e0c11f5d9721 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:39.092813 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18a902-6e68-4093-9f43-5478222c307d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:39.093459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18a902-6e68-4093-9f43-5478222c307d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:39.093736 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18a902-6e68-4093-9f43-5478222c307d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:39.094021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18a902-6e68-4093-9f43-5478222c307d None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:39.094778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18a902-6e68-4093-9f43-5478222c307d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:39.095063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18a902-6e68-4093-9f43-5478222c307d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:39.096560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18a902-6e68-4093-9f43-5478222c307d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:39.113016 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9eafad-77f4-4da4-b356-7223651e6613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:39.113649 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eafad-77f4-4da4-b356-7223651e6613 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:39.113942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eafad-77f4-4da4-b356-7223651e6613 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:39.114217 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eafad-77f4-4da4-b356-7223651e6613 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:39.116035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 322/652] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Aug 30 14:38:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:39.116851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18a902-6e68-4093-9f43-5478222c307d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:39.126424 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9eafad-77f4-4da4-b356-7223651e6613 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:39.127878 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9eafad-77f4-4da4-b356-7223651e6613 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:39.141360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-4d9eafad-77f4-4da4-b356-7223651e6613 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:38:39.141360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:39.141360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:39.141360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:39.141360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:39.141360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:39.141360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:39.141360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:39.141360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:39.141360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:39.141360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:39.141360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:39.141360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:39.141360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:39.141360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:39.141360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:39.142293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:39.143696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:39.143696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:39.143696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:39.143696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:39.143696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:39.143696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:39.143696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:39.143696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:39.143696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:38:39.143696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:39.143696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 331/653] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:39.143696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9eafad-77f4-4da4-b356-7223651e6613 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:39.223245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18a902-6e68-4093-9f43-5478222c307d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:39.234819 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:39.260432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9eafad-77f4-4da4-b356-7223651e6613 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:39.266852 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a5f1f-c478-4c3e-a989-850fd111721c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:39.267982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a5f1f-c478-4c3e-a989-850fd111721c None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:39.267982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a5f1f-c478-4c3e-a989-850fd111721c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:39.268148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a5f1f-c478-4c3e-a989-850fd111721c None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:39.268685 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a5f1f-c478-4c3e-a989-850fd111721c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:39.268989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a5f1f-c478-4c3e-a989-850fd111721c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:39.270358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a5f1f-c478-4c3e-a989-850fd111721c None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:39.286647 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 323/654] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:39.286647 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a5f1f-c478-4c3e-a989-850fd111721c None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:39.305239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0665f-6321-4007-88dc-795d83e1db6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:39.305915 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0665f-6321-4007-88dc-795d83e1db6d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:39.306261 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0665f-6321-4007-88dc-795d83e1db6d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:39.306597 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0665f-6321-4007-88dc-795d83e1db6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:39.308339 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0665f-6321-4007-88dc-795d83e1db6d None admin] RBAC: Adding query filter params (role.id=244ad93ecd1949c9a6e8f74b135b43b4, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=5b6e4c5bf2a54b28a55e5ae74b90ab4f) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:39.308339 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0665f-6321-4007-88dc-795d83e1db6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:39.309690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0665f-6321-4007-88dc-795d83e1db6d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:39.315607 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 332/655] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:38:39 2022] GET /identity/v3/role_assignments?user.id=5b6e4c5bf2a54b28a55e5ae74b90ab4f&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=244ad93ecd1949c9a6e8f74b135b43b4 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:39.316205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0665f-6321-4007-88dc-795d83e1db6d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:39.336941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:39.426432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a5f1f-c478-4c3e-a989-850fd111721c None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:39.452028 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0665f-6321-4007-88dc-795d83e1db6d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:39.470962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8149846-9640-4923-bb2e-cb995ef5b828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:39.471962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8149846-9640-4923-bb2e-cb995ef5b828 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:39.471962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8149846-9640-4923-bb2e-cb995ef5b828 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:39.471962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8149846-9640-4923-bb2e-cb995ef5b828 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:39.499927 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e7b28-56e1-40ac-880d-2678e2e99531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:39.500543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e7b28-56e1-40ac-880d-2678e2e99531 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:39.500819 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e7b28-56e1-40ac-880d-2678e2e99531 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:39.501090 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e7b28-56e1-40ac-880d-2678e2e99531 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:39.502254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e7b28-56e1-40ac-880d-2678e2e99531 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:39.502580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e7b28-56e1-40ac-880d-2678e2e99531 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:39.504400 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e7b28-56e1-40ac-880d-2678e2e99531 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:39.516096 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:39.525261 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 333/656] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Aug 30 14:38:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:39.526387 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e7b28-56e1-40ac-880d-2678e2e99531 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:39.558047 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8149846-9640-4923-bb2e-cb995ef5b828 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:39.559592 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8149846-9640-4923-bb2e-cb995ef5b828 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:39.598469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:39.646358 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e7b28-56e1-40ac-880d-2678e2e99531 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:39.681749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 324/657] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:39.681749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010361f-d3e5-4d46-8392-2ac026562631 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:39.681749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010361f-d3e5-4d46-8392-2ac026562631 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:39.681749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010361f-d3e5-4d46-8392-2ac026562631 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:39.681749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 325/658] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:39.681749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010361f-d3e5-4d46-8392-2ac026562631 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:39.695459 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ddbea3-d0e9-4623-9e39-e7f1f488ba88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:39.696087 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddbea3-d0e9-4623-9e39-e7f1f488ba88 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:39.696360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddbea3-d0e9-4623-9e39-e7f1f488ba88 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:39.696625 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddbea3-d0e9-4623-9e39-e7f1f488ba88 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:39.704601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ddbea3-d0e9-4623-9e39-e7f1f488ba88 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:39.706036 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ddbea3-d0e9-4623-9e39-e7f1f488ba88 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:39.718508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-e6ddbea3-d0e9-4623-9e39-e7f1f488ba88 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:38:39.718508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:39.718508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:39.718508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:39.718508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:39.718508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:39.718508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:39.718508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:39.718508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:39.718508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:39.718508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:39.718508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:39.718508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:39.718508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:39.718508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:39.718508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:39.719694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:39.720662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:39.720662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:39.720662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:39.720662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:39.720662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:39.720662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:39.720662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:39.720662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:39.720662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:38:39.720662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:39.720662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 334/659] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:39.720662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ddbea3-d0e9-4623-9e39-e7f1f488ba88 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:39.781234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:39.799619 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010361f-d3e5-4d46-8392-2ac026562631 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:39.825292 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ddbea3-d0e9-4623-9e39-e7f1f488ba88 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:39.835793 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade6564-8c22-4a31-acff-d8a7c0ebb140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:39.836478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade6564-8c22-4a31-acff-d8a7c0ebb140 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:39.836822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade6564-8c22-4a31-acff-d8a7c0ebb140 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:39.837160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade6564-8c22-4a31-acff-d8a7c0ebb140 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:39.847352 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:39.860956 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541894a6-844f-4784-aa86-fcef79ae84c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:39.861576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541894a6-844f-4784-aa86-fcef79ae84c0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:39.861852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541894a6-844f-4784-aa86-fcef79ae84c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:39.864085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541894a6-844f-4784-aa86-fcef79ae84c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:39.873313 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541894a6-844f-4784-aa86-fcef79ae84c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:39.874776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541894a6-844f-4784-aa86-fcef79ae84c0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:39.890428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-541894a6-844f-4784-aa86-fcef79ae84c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:39.890428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:39.890428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:39.890428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:39.890428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:39.890428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:39.890428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:39.890428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:39.890428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:39.890428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:39.890428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:39.890428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:39.890428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:39.890428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:39.890428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:39.890428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:39.891449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:39.892328 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:39.892328 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:39.892328 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:39.896442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 335/660] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:39.897207 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc94112-521e-4865-ac1c-52ebaa018d39 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:39.897463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc94112-521e-4865-ac1c-52ebaa018d39 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:39.897912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc94112-521e-4865-ac1c-52ebaa018d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:39.916400 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0dc94112-521e-4865-ac1c-52ebaa018d39 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:39.917726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0dc94112-521e-4865-ac1c-52ebaa018d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:39.924178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade6564-8c22-4a31-acff-d8a7c0ebb140 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:39.925618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade6564-8c22-4a31-acff-d8a7c0ebb140 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:39.986533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 336/661] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:39.992148 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc94112-521e-4865-ac1c-52ebaa018d39 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:40.030888 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 326/662] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:40.032012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade6564-8c22-4a31-acff-d8a7c0ebb140 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:40.042498 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:40.067584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:40.108084 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc94112-521e-4865-ac1c-52ebaa018d39 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:40.133090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade6564-8c22-4a31-acff-d8a7c0ebb140 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:40.148299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683ecbd-402d-4970-af3c-9c611b6795ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:40.149233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683ecbd-402d-4970-af3c-9c611b6795ed None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:40.149713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683ecbd-402d-4970-af3c-9c611b6795ed None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:40.150181 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683ecbd-402d-4970-af3c-9c611b6795ed None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:40.151599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683ecbd-402d-4970-af3c-9c611b6795ed None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:40.152107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683ecbd-402d-4970-af3c-9c611b6795ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:40.153760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683ecbd-402d-4970-af3c-9c611b6795ed None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:40.172930 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6e86e-051d-40e9-999d-8e49dd77f1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:40.175440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6e86e-051d-40e9-999d-8e49dd77f1ee None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:40.175440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6e86e-051d-40e9-999d-8e49dd77f1ee None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:40.175440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6e86e-051d-40e9-999d-8e49dd77f1ee None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:40.175440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6e86e-051d-40e9-999d-8e49dd77f1ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:40.175440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6e86e-051d-40e9-999d-8e49dd77f1ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:40.175440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6e86e-051d-40e9-999d-8e49dd77f1ee None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:40.181550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 337/663] 10.222.0.27 () {56 vars in 1139 bytes} [Tue Aug 30 14:38:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:40.183222 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683ecbd-402d-4970-af3c-9c611b6795ed None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:40.189464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 327/664] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:40.190346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6e86e-051d-40e9-999d-8e49dd77f1ee None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:40.286062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:40.291333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:40.293413 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683ecbd-402d-4970-af3c-9c611b6795ed None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:40.297530 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6e86e-051d-40e9-999d-8e49dd77f1ee None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:40.328133 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a8068-8a00-4a23-9305-94c4e2deeeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:40.328758 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a8068-8a00-4a23-9305-94c4e2deeeac None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:40.329073 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a8068-8a00-4a23-9305-94c4e2deeeac None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:40.329296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a8068-8a00-4a23-9305-94c4e2deeeac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:40.331363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d0cce-7c73-4b82-83bb-58edf13693f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:40.331964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d0cce-7c73-4b82-83bb-58edf13693f7 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:40.332241 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d0cce-7c73-4b82-83bb-58edf13693f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:40.332564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d0cce-7c73-4b82-83bb-58edf13693f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:40.340241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a8068-8a00-4a23-9305-94c4e2deeeac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:40.341683 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a8068-8a00-4a23-9305-94c4e2deeeac None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:40.358740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-c15a8068-8a00-4a23-9305-94c4e2deeeac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:40.358740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:40.358740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:40.358740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:40.358740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:40.358740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:40.358740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:40.358740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:40.358740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:40.358740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:40.358740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:40.358740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:40.358740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:40.358740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:40.358740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:40.358740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:40.359791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:40.360665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:40.360665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:40.360665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:40.360665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 338/665] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:40.360665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a8068-8a00-4a23-9305-94c4e2deeeac None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:40.406858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d0cce-7c73-4b82-83bb-58edf13693f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:40.408346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d0cce-7c73-4b82-83bb-58edf13693f7 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:40.466635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a8068-8a00-4a23-9305-94c4e2deeeac None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:40.502179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119f5ee-e243-47a1-a759-1039d83cd4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:40.502810 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119f5ee-e243-47a1-a759-1039d83cd4a0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:40.503086 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119f5ee-e243-47a1-a759-1039d83cd4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:40.503390 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119f5ee-e243-47a1-a759-1039d83cd4a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:40.512070 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119f5ee-e243-47a1-a759-1039d83cd4a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:40.513207 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:40.513556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119f5ee-e243-47a1-a759-1039d83cd4a0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:40.526385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 328/666] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:40.526385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d0cce-7c73-4b82-83bb-58edf13693f7 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:40.526615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-9119f5ee-e243-47a1-a759-1039d83cd4a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:40.526615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:40.526615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:40.526615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:40.526615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:40.526615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:40.526615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:40.526615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:40.526615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:40.526615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:40.526615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:40.526615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:40.526615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:40.526615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:40.526615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:40.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:40.529024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:40.529024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:40.529024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:40.529024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:40.529024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:40.529024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:40.529024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:40.529024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:40.529024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 339/667] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:40.529024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119f5ee-e243-47a1-a759-1039d83cd4a0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:40.538769 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:40.630201 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119f5ee-e243-47a1-a759-1039d83cd4a0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:40.642019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d0cce-7c73-4b82-83bb-58edf13693f7 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:40.665432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2aff3a-00ec-47ce-a80c-f040282c4d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:40.666191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2aff3a-00ec-47ce-a80c-f040282c4d37 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:40.666490 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2aff3a-00ec-47ce-a80c-f040282c4d37 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:40.666755 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2aff3a-00ec-47ce-a80c-f040282c4d37 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:40.668294 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2aff3a-00ec-47ce-a80c-f040282c4d37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:40.668601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2aff3a-00ec-47ce-a80c-f040282c4d37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:40.669982 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2aff3a-00ec-47ce-a80c-f040282c4d37 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:40.677997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76067ba4-5d1b-4c09-9b98-fd9f8afb1ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:40.678621 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76067ba4-5d1b-4c09-9b98-fd9f8afb1ba9 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:40.678894 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76067ba4-5d1b-4c09-9b98-fd9f8afb1ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:40.679193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76067ba4-5d1b-4c09-9b98-fd9f8afb1ba9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:40.682463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 340/668] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:40.683050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2aff3a-00ec-47ce-a80c-f040282c4d37 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:40.694269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76067ba4-5d1b-4c09-9b98-fd9f8afb1ba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:40.699156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-76067ba4-5d1b-4c09-9b98-fd9f8afb1ba9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:40.699156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:40.699156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:40.699156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:40.699156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:40.699156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:40.699156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:40.699156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:40.699156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:40.699156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:40.699156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:40.699156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:40.699156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:40.699156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:40.699156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:40.699156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:40.700118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:40.700118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:40.700118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:40.700118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:40.700118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:40.700118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:40.700118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:40.700118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:40.700118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:40.700118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:40.700118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:40.700118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:40.700118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:40.700118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:40.700118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:40.700118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:40.700968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:40.700968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:40.700968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:40.700968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:40.700968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:40.700968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:40.700968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:40.700968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:40.700968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:40.700968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:40.700968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:40.700968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:40.700968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 329/669] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:40.700968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76067ba4-5d1b-4c09-9b98-fd9f8afb1ba9 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:40.756520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:40.768501 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:40.794270 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2aff3a-00ec-47ce-a80c-f040282c4d37 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:40.818717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76067ba4-5d1b-4c09-9b98-fd9f8afb1ba9 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:40.835149 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5918dc-6e90-43e5-b00c-d66521a3b5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:40.835780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5918dc-6e90-43e5-b00c-d66521a3b5b9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:40.836061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5918dc-6e90-43e5-b00c-d66521a3b5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:40.836336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5918dc-6e90-43e5-b00c-d66521a3b5b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:40.846396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5918dc-6e90-43e5-b00c-d66521a3b5b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:40.847870 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5918dc-6e90-43e5-b00c-d66521a3b5b9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:40.853505 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a4f5f-e389-4093-a93c-29e5f24181c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:40.854106 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a4f5f-e389-4093-a93c-29e5f24181c9 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:40.854381 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a4f5f-e389-4093-a93c-29e5f24181c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:40.854684 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a4f5f-e389-4093-a93c-29e5f24181c9 None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:40.855399 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a4f5f-e389-4093-a93c-29e5f24181c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:40.855698 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a4f5f-e389-4093-a93c-29e5f24181c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:40.857110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a4f5f-e389-4093-a93c-29e5f24181c9 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:40.862736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-3b5918dc-6e90-43e5-b00c-d66521a3b5b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:40.862736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:40.862736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:40.862736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:40.862736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:40.862736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:40.862736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:40.862736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:40.862736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:40.862736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:40.862736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:40.862736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:40.862736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:40.862736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:40.862736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:40.862736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:40.863682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:40.864557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:40.864557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:40.864557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:40.864557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 341/670] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:40.864781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5918dc-6e90-43e5-b00c-d66521a3b5b9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:40.874078 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 330/671] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:40.875212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a4f5f-e389-4093-a93c-29e5f24181c9 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:40.972158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5918dc-6e90-43e5-b00c-d66521a3b5b9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:40.984007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a4f5f-e389-4093-a93c-29e5f24181c9 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:40.993488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:41.001099 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:41.008599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97d8a1-964e-4a7b-9ea4-ac2c58ef8951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:41.009259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97d8a1-964e-4a7b-9ea4-ac2c58ef8951 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:41.009553 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97d8a1-964e-4a7b-9ea4-ac2c58ef8951 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:41.009806 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97d8a1-964e-4a7b-9ea4-ac2c58ef8951 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:41.017113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97d8a1-964e-4a7b-9ea4-ac2c58ef8951 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:41.018609 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97d8a1-964e-4a7b-9ea4-ac2c58ef8951 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:41.018792 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc72e9-43d5-444e-95c4-9adada4fd8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:41.019467 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc72e9-43d5-444e-95c4-9adada4fd8b6 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:41.019744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc72e9-43d5-444e-95c4-9adada4fd8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:41.020012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc72e9-43d5-444e-95c4-9adada4fd8b6 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:41.020665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc72e9-43d5-444e-95c4-9adada4fd8b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:41.020964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc72e9-43d5-444e-95c4-9adada4fd8b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:41.022294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc72e9-43d5-444e-95c4-9adada4fd8b6 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:41.030356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-cd97d8a1-964e-4a7b-9ea4-ac2c58ef8951 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:41.030356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:41.030356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:41.030356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:41.030356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:41.030356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:41.030356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:41.030356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:41.030356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:41.030356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:41.030356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:41.030356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:41.030356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:41.030356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:41.030356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:41.031468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:41.037008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:41.037008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:41.037008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:41.037008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:41.037008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:41.037008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:41.037008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:41.037008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:41.037008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 342/672] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:41.037008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97d8a1-964e-4a7b-9ea4-ac2c58ef8951 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:41.037559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 331/673] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:41.037559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc72e9-43d5-444e-95c4-9adada4fd8b6 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:41.145355 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97d8a1-964e-4a7b-9ea4-ac2c58ef8951 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:41.147140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc72e9-43d5-444e-95c4-9adada4fd8b6 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:41.181731 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe66d4-d3c1-4f7f-8092-777945d24b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:41.182064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b5637-a25f-4858-a7fb-c6b61ff2e943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:41.182340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe66d4-d3c1-4f7f-8092-777945d24b28 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:41.182612 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b5637-a25f-4858-a7fb-c6b61ff2e943 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:41.182713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe66d4-d3c1-4f7f-8092-777945d24b28 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:41.183436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b5637-a25f-4858-a7fb-c6b61ff2e943 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:41.183533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe66d4-d3c1-4f7f-8092-777945d24b28 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:41.183710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b5637-a25f-4858-a7fb-c6b61ff2e943 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:41.184173 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe66d4-d3c1-4f7f-8092-777945d24b28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:41.184437 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b5637-a25f-4858-a7fb-c6b61ff2e943 None admin] RBAC: Adding query filter params (role.id=244ad93ecd1949c9a6e8f74b135b43b4, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=cfda7307e703427faaf599e32b91bc91) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:41.184539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe66d4-d3c1-4f7f-8092-777945d24b28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:41.184744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b5637-a25f-4858-a7fb-c6b61ff2e943 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:41.185838 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe66d4-d3c1-4f7f-8092-777945d24b28 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:41.186181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b5637-a25f-4858-a7fb-c6b61ff2e943 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:41.202376 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 343/674] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:41.202376 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe66d4-d3c1-4f7f-8092-777945d24b28 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:41.202625 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 332/675] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:38:41 2022] GET /identity/v3/role_assignments?user.id=cfda7307e703427faaf599e32b91bc91&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=244ad93ecd1949c9a6e8f74b135b43b4 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:41.202625 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b5637-a25f-4858-a7fb-c6b61ff2e943 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:41.254864 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:41.267378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:41.331456 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b5637-a25f-4858-a7fb-c6b61ff2e943 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:41.346250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe66d4-d3c1-4f7f-8092-777945d24b28 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:41.375690 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0483285-9766-4901-b124-fa379d446a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:41.376305 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0483285-9766-4901-b124-fa379d446a85 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:41.376586 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0483285-9766-4901-b124-fa379d446a85 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:41.376859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0483285-9766-4901-b124-fa379d446a85 None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:41.378409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0483285-9766-4901-b124-fa379d446a85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:41.378714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0483285-9766-4901-b124-fa379d446a85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:41.380224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0483285-9766-4901-b124-fa379d446a85 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:41.385784 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0912080-c197-4f9a-87e4-62742d1d2fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:41.386536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0912080-c197-4f9a-87e4-62742d1d2fc5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:41.386714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0912080-c197-4f9a-87e4-62742d1d2fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:41.386964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0912080-c197-4f9a-87e4-62742d1d2fc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:41.393543 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 333/676] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:41.394934 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0483285-9766-4901-b124-fa379d446a85 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:41.398115 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0912080-c197-4f9a-87e4-62742d1d2fc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:41.400669 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0912080-c197-4f9a-87e4-62742d1d2fc5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:41.418698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-b0912080-c197-4f9a-87e4-62742d1d2fc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:41.418698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:41.418698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:41.418698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:41.418698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:41.418698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:41.418698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:41.418698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:41.418698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:41.418698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:41.418698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:41.418698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:41.418698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:41.418698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:41.418698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:41.418698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:41.419745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:41.420643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:41.420643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:41.420643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:41.420643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 344/677] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:41.422098 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea24dd-41c4-44f8-93a9-59d845fb2764 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:41.422098 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea24dd-41c4-44f8-93a9-59d845fb2764 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:41.422098 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea24dd-41c4-44f8-93a9-59d845fb2764 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:41.423388 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 345/678] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:41.424006 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea24dd-41c4-44f8-93a9-59d845fb2764 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:41.516854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0483285-9766-4901-b124-fa379d446a85 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:41.519060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:41.523061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:41.533334 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea24dd-41c4-44f8-93a9-59d845fb2764 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:41.553318 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cccd46-4c68-420c-a09b-50a60f6e6a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:41.553948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cccd46-4c68-420c-a09b-50a60f6e6a70 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:41.554247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cccd46-4c68-420c-a09b-50a60f6e6a70 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:41.554516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cccd46-4c68-420c-a09b-50a60f6e6a70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:41.561073 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cccd46-4c68-420c-a09b-50a60f6e6a70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:41.562509 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cccd46-4c68-420c-a09b-50a60f6e6a70 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:41.568890 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac1afd-4e38-4430-9017-5d582fc544df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:41.569633 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac1afd-4e38-4430-9017-5d582fc544df None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:41.576359 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac1afd-4e38-4430-9017-5d582fc544df None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:41.576359 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac1afd-4e38-4430-9017-5d582fc544df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:41.576359 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac1afd-4e38-4430-9017-5d582fc544df None admin] RBAC: Adding query filter params (role.id=244ad93ecd1949c9a6e8f74b135b43b4, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=ef389656cc8546fdb7d9f30329b93dbd) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:41.576359 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac1afd-4e38-4430-9017-5d582fc544df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:41.576359 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac1afd-4e38-4430-9017-5d582fc544df None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:41.576723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-e7cccd46-4c68-420c-a09b-50a60f6e6a70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:41.576723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:41.576723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:41.576723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:41.576723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:41.576723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:41.576723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:41.576723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:41.576723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:41.576723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:41.576723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:41.576723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:41.576723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:41.576723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:41.576723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:41.577554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:41.578464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:41.578464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:41.578464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:41.578464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:41.578464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:41.578464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:41.578464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:41.578464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:41.578464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 334/679] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:41.578464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cccd46-4c68-420c-a09b-50a60f6e6a70 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:41.583334 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 346/680] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:38:41 2022] GET /identity/v3/role_assignments?user.id=ef389656cc8546fdb7d9f30329b93dbd&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=244ad93ecd1949c9a6e8f74b135b43b4 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:41.584455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c0cc6-0f46-4730-9bd0-b6fd10c31a5d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:41.584806 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c0cc6-0f46-4730-9bd0-b6fd10c31a5d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:41.585080 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c0cc6-0f46-4730-9bd0-b6fd10c31a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:41.605195 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-cf0c0cc6-0f46-4730-9bd0-b6fd10c31a5d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:41.606410 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-cf0c0cc6-0f46-4730-9bd0-b6fd10c31a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:41.699637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 347/681] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:41.701022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c0cc6-0f46-4730-9bd0-b6fd10c31a5d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:41.726541 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cccd46-4c68-420c-a09b-50a60f6e6a70 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:41.784485 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2534ba20-0aa3-4647-8f85-378f04b4ca16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:41.785417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534ba20-0aa3-4647-8f85-378f04b4ca16 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:41.785997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534ba20-0aa3-4647-8f85-378f04b4ca16 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:41.786459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534ba20-0aa3-4647-8f85-378f04b4ca16 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:41.787808 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2534ba20-0aa3-4647-8f85-378f04b4ca16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:41.788290 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2534ba20-0aa3-4647-8f85-378f04b4ca16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:41.790103 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2534ba20-0aa3-4647-8f85-378f04b4ca16 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:41.806187 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 335/682] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:41.810640 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:41.811665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2534ba20-0aa3-4647-8f85-378f04b4ca16 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:41.831081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c0cc6-0f46-4730-9bd0-b6fd10c31a5d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:41.844475 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:41.887019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b76bda-cf20-4345-8adb-e925b2f3454f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:41.887707 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b76bda-cf20-4345-8adb-e925b2f3454f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:41.887984 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b76bda-cf20-4345-8adb-e925b2f3454f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:41.888253 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b76bda-cf20-4345-8adb-e925b2f3454f None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:41.889295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b76bda-cf20-4345-8adb-e925b2f3454f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:41.889601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b76bda-cf20-4345-8adb-e925b2f3454f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:41.891511 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b76bda-cf20-4345-8adb-e925b2f3454f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:41.907876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 348/683] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:41.917090 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b76bda-cf20-4345-8adb-e925b2f3454f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:41.941422 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2534ba20-0aa3-4647-8f85-378f04b4ca16 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:41.984119 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d98ea2-d1eb-4959-87ca-458eddf1169e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:41.984748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d98ea2-d1eb-4959-87ca-458eddf1169e None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:41.985029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d98ea2-d1eb-4959-87ca-458eddf1169e None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:41.985315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d98ea2-d1eb-4959-87ca-458eddf1169e None admin] PATH_INFO: `/v3/users/swift` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:41.994764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d98ea2-d1eb-4959-87ca-458eddf1169e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:42.003119 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d98ea2-d1eb-4959-87ca-458eddf1169e None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:42.021891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-14d98ea2-d1eb-4959-87ca-458eddf1169e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:38:42.021891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:42.021891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:42.021891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:42.021891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:42.021891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:42.021891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:42.021891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:42.021891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:42.021891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:42.021891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:42.021891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:42.021891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:42.021891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:42.021891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:42.021891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:42.022853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:42.023840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:42.023840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:42.023840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:42.023840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:42.023840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:42.023840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:42.023840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:42.023840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:42.023840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:38:42.023840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:42.023840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 336/684] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:38:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:42.025055 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d98ea2-d1eb-4959-87ca-458eddf1169e None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:42.053261 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b76bda-cf20-4345-8adb-e925b2f3454f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:42.089806 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f4f9f-f75a-4cda-8220-8e7fd4b86537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:42.090747 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f4f9f-f75a-4cda-8220-8e7fd4b86537 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:42.091275 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f4f9f-f75a-4cda-8220-8e7fd4b86537 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:42.091775 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f4f9f-f75a-4cda-8220-8e7fd4b86537 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:42.092767 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f4f9f-f75a-4cda-8220-8e7fd4b86537 None admin] RBAC: Adding query filter params (role.id=244ad93ecd1949c9a6e8f74b135b43b4, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=7bff0664e0b44b7c85344a9f3e88f126) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:42.093319 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f4f9f-f75a-4cda-8220-8e7fd4b86537 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:42.094941 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f4f9f-f75a-4cda-8220-8e7fd4b86537 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:42.101556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 349/685] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:38:41 2022] GET /identity/v3/role_assignments?user.id=7bff0664e0b44b7c85344a9f3e88f126&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=244ad93ecd1949c9a6e8f74b135b43b4 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:42.108273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f4f9f-f75a-4cda-8220-8e7fd4b86537 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:42.110024 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:42.131275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d98ea2-d1eb-4959-87ca-458eddf1169e None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:42.143525 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:42.212605 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de53b1e-0b6d-460f-b38d-45db1a14b70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:42.213297 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de53b1e-0b6d-460f-b38d-45db1a14b70f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:42.213576 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de53b1e-0b6d-460f-b38d-45db1a14b70f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:42.214380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de53b1e-0b6d-460f-b38d-45db1a14b70f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:42.235850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de53b1e-0b6d-460f-b38d-45db1a14b70f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:42.241096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-4de53b1e-0b6d-460f-b38d-45db1a14b70f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:42.241096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:42.241096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:42.241096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:42.241096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:42.241096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:42.241096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:42.241096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:42.241096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:42.241096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:42.241096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:42.241096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:42.241096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:42.241096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:42.241096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:42.241096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:42.242076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:42.242076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:42.242076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:42.242076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:42.242076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:42.242076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:42.242076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:42.242076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:42.242076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:42.242076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:42.242076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:42.242076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:42.242076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:42.242076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:42.242076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:42.242076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:42.242922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:42.242922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:42.242922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:42.242922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:42.242922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:42.242922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:42.242922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:42.242922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:42.242922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:42.242922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:42.242922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:42.242922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:42.242922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 337/686] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:38:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:42.248502 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de53b1e-0b6d-460f-b38d-45db1a14b70f None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:42.293259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f4f9f-f75a-4cda-8220-8e7fd4b86537 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:42.346079 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4279d8d-689d-4308-849a-c4abb5c7918c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:42.346866 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4279d8d-689d-4308-849a-c4abb5c7918c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:42.347335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4279d8d-689d-4308-849a-c4abb5c7918c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:42.347609 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4279d8d-689d-4308-849a-c4abb5c7918c None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:42.348429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4279d8d-689d-4308-849a-c4abb5c7918c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:42.348738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4279d8d-689d-4308-849a-c4abb5c7918c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:42.350478 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4279d8d-689d-4308-849a-c4abb5c7918c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:42.371605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 350/687] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Aug 30 14:38:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:42.377018 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4279d8d-689d-4308-849a-c4abb5c7918c None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:42.425439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de53b1e-0b6d-460f-b38d-45db1a14b70f None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:42.455623 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:42.467749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5488709-91ff-4110-96ab-054e955cfbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:42.470247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5488709-91ff-4110-96ab-054e955cfbc3 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:42.470247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5488709-91ff-4110-96ab-054e955cfbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:42.470247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5488709-91ff-4110-96ab-054e955cfbc3 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:42.470247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5488709-91ff-4110-96ab-054e955cfbc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:42.470247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5488709-91ff-4110-96ab-054e955cfbc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:42.471375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5488709-91ff-4110-96ab-054e955cfbc3 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:42.481158 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 338/688] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:42.488777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5488709-91ff-4110-96ab-054e955cfbc3 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:42.494560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:42.496383 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4279d8d-689d-4308-849a-c4abb5c7918c None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:42.575744 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e402d-9d3b-4772-8d5e-4015f709a932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:42.576416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e402d-9d3b-4772-8d5e-4015f709a932 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:42.576643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e402d-9d3b-4772-8d5e-4015f709a932 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:42.576886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e402d-9d3b-4772-8d5e-4015f709a932 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:42.603007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5488709-91ff-4110-96ab-054e955cfbc3 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:42.641809 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca845738-43f6-42f0-97cb-4a1485f2d54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:42.642587 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca845738-43f6-42f0-97cb-4a1485f2d54f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:42.642869 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca845738-43f6-42f0-97cb-4a1485f2d54f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:42.643164 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca845738-43f6-42f0-97cb-4a1485f2d54f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:42.656369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca845738-43f6-42f0-97cb-4a1485f2d54f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:42.658103 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca845738-43f6-42f0-97cb-4a1485f2d54f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:42.674273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-ca845738-43f6-42f0-97cb-4a1485f2d54f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:42.674273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:42.674273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:42.674273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:42.674273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:42.674273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:42.674273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:42.674273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:42.674273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:42.674273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:42.674273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:42.674273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:42.674273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:42.674273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:42.674273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:42.674273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:42.675254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:42.676167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:42.676167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:42.676167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:42.676167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 339/689] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:42.682653 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca845738-43f6-42f0-97cb-4a1485f2d54f None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:42.727201 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e402d-9d3b-4772-8d5e-4015f709a932 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:42.728667 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e402d-9d3b-4772-8d5e-4015f709a932 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:42.813688 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:42.815941 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:42.862268 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 351/690] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:42.867533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e402d-9d3b-4772-8d5e-4015f709a932 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:42.895736 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca845738-43f6-42f0-97cb-4a1485f2d54f None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:42.935732 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa769c3-6aa0-471d-bdec-cdad75811359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:42.936338 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa769c3-6aa0-471d-bdec-cdad75811359 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:42.936617 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa769c3-6aa0-471d-bdec-cdad75811359 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:42.936889 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa769c3-6aa0-471d-bdec-cdad75811359 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:42.937536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa769c3-6aa0-471d-bdec-cdad75811359 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:42.939488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa769c3-6aa0-471d-bdec-cdad75811359 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:42.941227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa769c3-6aa0-471d-bdec-cdad75811359 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:42.962343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 340/691] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:42.968145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa769c3-6aa0-471d-bdec-cdad75811359 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:43.023870 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e402d-9d3b-4772-8d5e-4015f709a932 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:43.071336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae337ba-5229-45e7-bf87-688a6e626da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:43.071904 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae337ba-5229-45e7-bf87-688a6e626da4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:43.072185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae337ba-5229-45e7-bf87-688a6e626da4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:43.072456 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae337ba-5229-45e7-bf87-688a6e626da4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:43.095724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae337ba-5229-45e7-bf87-688a6e626da4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:43.097189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae337ba-5229-45e7-bf87-688a6e626da4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:43.127960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-8ae337ba-5229-45e7-bf87-688a6e626da4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:43.127960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:43.127960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:43.127960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:43.127960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:43.127960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:43.127960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:43.127960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:43.127960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:43.127960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:43.127960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:43.127960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:43.127960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:43.127960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:43.127960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:43.127960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:43.129008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:43.129875 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:43.129875 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:43.129875 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:43.129875 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 352/692] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:43.135349 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae337ba-5229-45e7-bf87-688a6e626da4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:43.152970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa769c3-6aa0-471d-bdec-cdad75811359 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:43.182167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:43.188133 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:43.208687 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04432847-a02a-4f65-93be-77ee13c01d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:43.209317 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04432847-a02a-4f65-93be-77ee13c01d81 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:43.209597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04432847-a02a-4f65-93be-77ee13c01d81 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:43.209946 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04432847-a02a-4f65-93be-77ee13c01d81 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:43.224586 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04432847-a02a-4f65-93be-77ee13c01d81 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:43.227608 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04432847-a02a-4f65-93be-77ee13c01d81 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:43.253499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-04432847-a02a-4f65-93be-77ee13c01d81 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:38:43.253499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:43.253499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:43.253499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:43.253499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:43.253499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:43.253499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:43.253499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:43.253499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:43.253499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:43.253499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:43.253499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:43.253499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:43.253499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:43.253499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:43.253499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:43.254477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:43.255449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:43.255449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:43.255449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:43.255449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:43.255449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:43.255449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:43.255449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:43.255449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:43.255449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:38:43.255449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:43.262192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 341/693] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 30 14:38:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:38:43.272361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04432847-a02a-4f65-93be-77ee13c01d81 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:43.326891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae337ba-5229-45e7-bf87-688a6e626da4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:43.387061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf32b75-9f7d-4040-97a5-8d1b8713bbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:43.389005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf32b75-9f7d-4040-97a5-8d1b8713bbf5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:43.389514 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf32b75-9f7d-4040-97a5-8d1b8713bbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:43.391012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf32b75-9f7d-4040-97a5-8d1b8713bbf5 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:43.391911 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf32b75-9f7d-4040-97a5-8d1b8713bbf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:43.392599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf32b75-9f7d-4040-97a5-8d1b8713bbf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:43.395557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf32b75-9f7d-4040-97a5-8d1b8713bbf5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:43.424079 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 353/694] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:43.426714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf32b75-9f7d-4040-97a5-8d1b8713bbf5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:43.459366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04432847-a02a-4f65-93be-77ee13c01d81 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:43.508758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0834d-7165-47ee-8211-bff38365caf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:43.510326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0834d-7165-47ee-8211-bff38365caf7 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:43.511395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0834d-7165-47ee-8211-bff38365caf7 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:43.512128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0834d-7165-47ee-8211-bff38365caf7 None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:43.513368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0834d-7165-47ee-8211-bff38365caf7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:43.513873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0834d-7165-47ee-8211-bff38365caf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:43.516366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0834d-7165-47ee-8211-bff38365caf7 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:43.527272 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:43.540571 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 342/695] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 30 14:38:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:43.541962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f0a9b-82eb-4661-8798-e96302aa27b3 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:43.542317 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f0a9b-82eb-4661-8798-e96302aa27b3 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:43.542659 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f0a9b-82eb-4661-8798-e96302aa27b3 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:43.544236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 343/696] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:43.550898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc787-d4c4-4362-9f19-6b2ca523047a None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:43.550898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc787-d4c4-4362-9f19-6b2ca523047a None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:43.551232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc787-d4c4-4362-9f19-6b2ca523047a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:43.554050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf32b75-9f7d-4040-97a5-8d1b8713bbf5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:43.574070 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-94ddc787-d4c4-4362-9f19-6b2ca523047a None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:43.575905 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-94ddc787-d4c4-4362-9f19-6b2ca523047a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:43.590736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5ba11-32ff-4f43-8c83-a1db99feeb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:43.591553 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5ba11-32ff-4f43-8c83-a1db99feeb9d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:43.591972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5ba11-32ff-4f43-8c83-a1db99feeb9d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:43.592374 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5ba11-32ff-4f43-8c83-a1db99feeb9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:43.600538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5ba11-32ff-4f43-8c83-a1db99feeb9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:43.602158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5ba11-32ff-4f43-8c83-a1db99feeb9d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:43.613095 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:43.613942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-0ad5ba11-32ff-4f43-8c83-a1db99feeb9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:43.613942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:43.613942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:43.613942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:43.613942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:43.613942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:43.613942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:43.613942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:43.613942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:43.613942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:43.613942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:43.613942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:43.613942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:43.613942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:43.613942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:43.614891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:43.616017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:43.616017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:43.616017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:43.616017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:43.616017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:43.616017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:43.616017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:43.616017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:43.616017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 354/697] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:43.616637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5ba11-32ff-4f43-8c83-a1db99feeb9d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:43.667603 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 344/698] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:43.668996 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc787-d4c4-4362-9f19-6b2ca523047a None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:43.749287 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5ba11-32ff-4f43-8c83-a1db99feeb9d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:43.794180 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc787-d4c4-4362-9f19-6b2ca523047a None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:43.831774 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d267a0d-aecd-4a2c-bf6c-deae8bf384f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:43.833124 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d267a0d-aecd-4a2c-bf6c-deae8bf384f8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:43.834924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d267a0d-aecd-4a2c-bf6c-deae8bf384f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:43.835386 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d267a0d-aecd-4a2c-bf6c-deae8bf384f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:43.839238 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fe020-941b-4f96-bc3b-3ac23254be0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:43.840342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fe020-941b-4f96-bc3b-3ac23254be0e None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:43.840829 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fe020-941b-4f96-bc3b-3ac23254be0e None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:43.842213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fe020-941b-4f96-bc3b-3ac23254be0e None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:43.843059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fe020-941b-4f96-bc3b-3ac23254be0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:43.846219 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fe020-941b-4f96-bc3b-3ac23254be0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:43.847839 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fe020-941b-4f96-bc3b-3ac23254be0e None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:43.861255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:43.865537 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 345/699] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:43.876110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fe020-941b-4f96-bc3b-3ac23254be0e None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:43.924347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d267a0d-aecd-4a2c-bf6c-deae8bf384f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:43.924947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:43.926163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d267a0d-aecd-4a2c-bf6c-deae8bf384f8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:43.986383 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fe020-941b-4f96-bc3b-3ac23254be0e None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:44.021968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4739f5-441e-4a7f-ab99-f3c317eb3088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:44.022565 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4739f5-441e-4a7f-ab99-f3c317eb3088 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:44.022841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4739f5-441e-4a7f-ab99-f3c317eb3088 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:44.023119 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4739f5-441e-4a7f-ab99-f3c317eb3088 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:44.023854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4739f5-441e-4a7f-ab99-f3c317eb3088 None admin] RBAC: Adding query filter params (role.id=244ad93ecd1949c9a6e8f74b135b43b4, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=bc23480d5aa94cad9e6599762ce01548) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:44.024160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4739f5-441e-4a7f-ab99-f3c317eb3088 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:44.025833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4739f5-441e-4a7f-ab99-f3c317eb3088 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:44.039139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 355/700] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:44.039139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fc66a-5793-4445-8a58-91e34705e50e None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:44.039139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fc66a-5793-4445-8a58-91e34705e50e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:44.039139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fc66a-5793-4445-8a58-91e34705e50e None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:44.039139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 356/701] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:44.039139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fc66a-5793-4445-8a58-91e34705e50e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:44.039682 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 346/702] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:38:43 2022] GET /identity/v3/role_assignments?user.id=bc23480d5aa94cad9e6599762ce01548&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=244ad93ecd1949c9a6e8f74b135b43b4 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:44.040734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b7614-9d18-4dc0-a913-9b56c3e9a549 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:44.041020 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b7614-9d18-4dc0-a913-9b56c3e9a549 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:44.041293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b7614-9d18-4dc0-a913-9b56c3e9a549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:44.061602 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-638b7614-9d18-4dc0-a913-9b56c3e9a549 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:44.062832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-638b7614-9d18-4dc0-a913-9b56c3e9a549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:44.105414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:44.159942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 347/703] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:44.159942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b7614-9d18-4dc0-a913-9b56c3e9a549 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:44.189732 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:44.204318 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fc66a-5793-4445-8a58-91e34705e50e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:44.246907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa524260-0388-462d-895e-8440dc6711df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:44.247871 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa524260-0388-462d-895e-8440dc6711df None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:44.248159 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa524260-0388-462d-895e-8440dc6711df None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:44.248488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa524260-0388-462d-895e-8440dc6711df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:44.263695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa524260-0388-462d-895e-8440dc6711df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:44.266145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa524260-0388-462d-895e-8440dc6711df None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:44.289199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-aa524260-0388-462d-895e-8440dc6711df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:44.289199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:44.289199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:44.289199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:44.289199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:44.289199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:44.289199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:44.289199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:44.289199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:44.289199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:44.289199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:44.289199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:44.289199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:44.289199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:44.289199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:44.289199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:44.290467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:44.291600 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:44.291600 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:44.291600 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:44.291600 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 357/704] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:44.292400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b7614-9d18-4dc0-a913-9b56c3e9a549 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:44.298263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa524260-0388-462d-895e-8440dc6711df None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:44.340209 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c75bca-dd5e-4e5a-a76a-5084b24b1b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:44.340835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c75bca-dd5e-4e5a-a76a-5084b24b1b40 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:44.341114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c75bca-dd5e-4e5a-a76a-5084b24b1b40 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:44.341382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c75bca-dd5e-4e5a-a76a-5084b24b1b40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:44.354838 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c75bca-dd5e-4e5a-a76a-5084b24b1b40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:44.356566 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c75bca-dd5e-4e5a-a76a-5084b24b1b40 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:44.372192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-f2c75bca-dd5e-4e5a-a76a-5084b24b1b40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:44.372192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:44.372192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:44.372192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:44.372192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:44.372192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:44.372192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:44.372192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:44.372192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:44.372192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:44.372192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:44.372192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:44.372192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:44.372192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:44.372192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:44.372192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:44.373153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:44.374050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:44.374050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:44.374050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:44.374050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 348/705] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:44.380485 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c75bca-dd5e-4e5a-a76a-5084b24b1b40 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:44.425162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:44.438374 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa524260-0388-462d-895e-8440dc6711df None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:44.479198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:44.484131 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7d713-2f82-47ce-9a73-deda8b8c28c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:44.484742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7d713-2f82-47ce-9a73-deda8b8c28c7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:44.485078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7d713-2f82-47ce-9a73-deda8b8c28c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:44.485352 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7d713-2f82-47ce-9a73-deda8b8c28c7 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:44.486163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7d713-2f82-47ce-9a73-deda8b8c28c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:44.486468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7d713-2f82-47ce-9a73-deda8b8c28c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:44.488318 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7d713-2f82-47ce-9a73-deda8b8c28c7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:44.502866 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 358/706] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:44.508371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7d713-2f82-47ce-9a73-deda8b8c28c7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:44.518323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c75bca-dd5e-4e5a-a76a-5084b24b1b40 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:44.569173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05488b43-82ef-4104-ae8b-274df5b00937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:44.570589 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05488b43-82ef-4104-ae8b-274df5b00937 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:44.571198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05488b43-82ef-4104-ae8b-274df5b00937 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:44.571742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05488b43-82ef-4104-ae8b-274df5b00937 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:44.572940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05488b43-82ef-4104-ae8b-274df5b00937 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:44.574438 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05488b43-82ef-4104-ae8b-274df5b00937 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:44.576310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05488b43-82ef-4104-ae8b-274df5b00937 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:44.593279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 349/707] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:44.594406 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05488b43-82ef-4104-ae8b-274df5b00937 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:44.621099 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7d713-2f82-47ce-9a73-deda8b8c28c7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:44.657475 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56edc6-7e5b-40fb-a9e3-b6ce9c8f9257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:44.658076 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56edc6-7e5b-40fb-a9e3-b6ce9c8f9257 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:44.658438 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56edc6-7e5b-40fb-a9e3-b6ce9c8f9257 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:44.658716 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56edc6-7e5b-40fb-a9e3-b6ce9c8f9257 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:44.659421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56edc6-7e5b-40fb-a9e3-b6ce9c8f9257 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:44.660857 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56edc6-7e5b-40fb-a9e3-b6ce9c8f9257 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:44.702643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05488b43-82ef-4104-ae8b-274df5b00937 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:44.731483 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:44.737229 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6608ddc-f41a-4baa-a52c-bc30551a5358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:44.737855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6608ddc-f41a-4baa-a52c-bc30551a5358 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:44.738135 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6608ddc-f41a-4baa-a52c-bc30551a5358 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:44.738413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6608ddc-f41a-4baa-a52c-bc30551a5358 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:44.739003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6608ddc-f41a-4baa-a52c-bc30551a5358 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:44.740396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6608ddc-f41a-4baa-a52c-bc30551a5358 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-b6608ddc-f41a-4baa-a52c-bc30551a5358 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:44.751255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:44.752330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:44.752330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:44.752330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:44.752330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:44.752330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:44.752330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:44.752330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:38:44.752330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:38:44.752330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:38:44.752330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:38:44.752330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:44.752330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:44.752330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:44.752330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:44.752330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:44.752330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:44.753198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:44.753198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:44.753198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:44.753198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:44.753198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:44.753198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:44.753198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:44.753198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:44.753198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:44.753198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:44.753198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:38:44.753198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:38:44.753198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:38:44.753198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:44.753198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 350/708] 10.222.0.27 () {56 vars in 1129 bytes} [Tue Aug 30 14:38:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:44.753998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b990d6-eb2d-4ec4-a64d-320d842632d3 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:44.753998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b990d6-eb2d-4ec4-a64d-320d842632d3 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:44.753998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b990d6-eb2d-4ec4-a64d-320d842632d3 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:44.755361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 351/709] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:44.755882 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b990d6-eb2d-4ec4-a64d-320d842632d3 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:44.774555 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 359/710] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:38:44 2022] POST /identity/v3/users => generated 254 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:38:44.784268 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56edc6-7e5b-40fb-a9e3-b6ce9c8f9257 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:44.807443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:44.886890 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b990d6-eb2d-4ec4-a64d-320d842632d3 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:44.929966 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e9454-35b8-4129-b32e-ed7eda4d49ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:44.930600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e9454-35b8-4129-b32e-ed7eda4d49ea None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:44.930878 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e9454-35b8-4129-b32e-ed7eda4d49ea None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:44.931596 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e9454-35b8-4129-b32e-ed7eda4d49ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:44.934637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56edc6-7e5b-40fb-a9e3-b6ce9c8f9257 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:44.941015 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e9454-35b8-4129-b32e-ed7eda4d49ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:44.941015 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e9454-35b8-4129-b32e-ed7eda4d49ea None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:44.953113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-454e9454-35b8-4129-b32e-ed7eda4d49ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:44.953113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:44.953113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:44.953113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:44.953113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:44.953113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:44.953113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:44.953113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:44.953113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:44.953113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:44.953113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:44.953113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:44.953113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:44.953113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:44.953113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:44.954023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:44.954944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:44.954944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:44.954944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:44.954944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:44.954944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:44.954944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:44.954944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:44.954944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:44.954944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 352/711] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:44.956284 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718b9d3-243d-41ea-ae58-046fa198a15f None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:44.956284 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718b9d3-243d-41ea-ae58-046fa198a15f None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:44.956284 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718b9d3-243d-41ea-ae58-046fa198a15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:44.977002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4500be7-540f-4a86-bb77-09dd39ad382a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:44.977379 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-2718b9d3-243d-41ea-ae58-046fa198a15f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:44.977617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4500be7-540f-4a86-bb77-09dd39ad382a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:44.977900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4500be7-540f-4a86-bb77-09dd39ad382a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:44.978174 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4500be7-540f-4a86-bb77-09dd39ad382a None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:44.978571 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-2718b9d3-243d-41ea-ae58-046fa198a15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:44.978811 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4500be7-540f-4a86-bb77-09dd39ad382a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:44.980575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4500be7-540f-4a86-bb77-09dd39ad382a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:44.981133 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4500be7-540f-4a86-bb77-09dd39ad382a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:44.987785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 360/712] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Aug 30 14:38:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:44.989190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4500be7-540f-4a86-bb77-09dd39ad382a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:45.055128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:45.075295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:45.081458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 353/713] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:45.083083 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718b9d3-243d-41ea-ae58-046fa198a15f None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:45.107648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4500be7-540f-4a86-bb77-09dd39ad382a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:45.155138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8ba7f-b6b1-41d7-9438-bec4312ef34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:45.161125 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8ba7f-b6b1-41d7-9438-bec4312ef34c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:45.161406 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8ba7f-b6b1-41d7-9438-bec4312ef34c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:45.168827 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8ba7f-b6b1-41d7-9438-bec4312ef34c None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:45.169740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8ba7f-b6b1-41d7-9438-bec4312ef34c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:45.170057 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8ba7f-b6b1-41d7-9438-bec4312ef34c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:45.171542 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8ba7f-b6b1-41d7-9438-bec4312ef34c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:45.197355 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 361/714] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:45.204190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8ba7f-b6b1-41d7-9438-bec4312ef34c None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:45.401811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718b9d3-243d-41ea-ae58-046fa198a15f None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:45.410737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8ba7f-b6b1-41d7-9438-bec4312ef34c None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:45.445918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:45.447783 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49abe0-7a9b-4b0c-9fe6-adfb5c3bf24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:45.448429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49abe0-7a9b-4b0c-9fe6-adfb5c3bf24b None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:45.448781 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49abe0-7a9b-4b0c-9fe6-adfb5c3bf24b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:45.448981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49abe0-7a9b-4b0c-9fe6-adfb5c3bf24b None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:45.449624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49abe0-7a9b-4b0c-9fe6-adfb5c3bf24b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:45.449924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49abe0-7a9b-4b0c-9fe6-adfb5c3bf24b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:45.451314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49abe0-7a9b-4b0c-9fe6-adfb5c3bf24b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:45.451796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac7a4e-b248-427a-8b45-a418f552f4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:45.452757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac7a4e-b248-427a-8b45-a418f552f4cc None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:45.453237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac7a4e-b248-427a-8b45-a418f552f4cc None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:45.453779 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac7a4e-b248-427a-8b45-a418f552f4cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:45.454978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac7a4e-b248-427a-8b45-a418f552f4cc None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=5b6e4c5bf2a54b28a55e5ae74b90ab4f) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:45.455538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac7a4e-b248-427a-8b45-a418f552f4cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:45.457111 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac7a4e-b248-427a-8b45-a418f552f4cc None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:45.463423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 354/715] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Aug 30 14:38:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:45.463423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49abe0-7a9b-4b0c-9fe6-adfb5c3bf24b None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:45.471139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 362/716] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:38:45 2022] GET /identity/v3/role_assignments?user.id=5b6e4c5bf2a54b28a55e5ae74b90ab4f&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:45.546869 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:45.645655 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49abe0-7a9b-4b0c-9fe6-adfb5c3bf24b None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:45.687550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6711bd1-80f9-4895-8e38-00c6f953f5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:45.688535 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6711bd1-80f9-4895-8e38-00c6f953f5ea None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:45.689020 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6711bd1-80f9-4895-8e38-00c6f953f5ea None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:45.689496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6711bd1-80f9-4895-8e38-00c6f953f5ea None admin] PATH_INFO: `/v3/services/network` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:45.690303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6711bd1-80f9-4895-8e38-00c6f953f5ea None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:45.691918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6711bd1-80f9-4895-8e38-00c6f953f5ea None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-e6711bd1-80f9-4895-8e38-00c6f953f5ea None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:45.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:45.703774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:45.703774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:45.703774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:45.703774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:45.703774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:45.703774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:45.703774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:38:45.703774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:38:45.703774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:38:45.703774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:38:45.703774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:45.703774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:45.703774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:45.703774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:45.703774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:45.703774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:45.704624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:45.704624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:45.704624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:45.704624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:45.704624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:45.704624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:45.704624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:45.704624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:45.704624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:45.704624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:45.704624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:38:45.704624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:38:45.704624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:38:45.704624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:45.707346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 355/717] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:45 2022] GET /identity/v3/services/network => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:45.715984 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac7a4e-b248-427a-8b45-a418f552f4cc None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:45.825381 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac7a4e-b248-427a-8b45-a418f552f4cc None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:45.861956 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1108b-694c-431b-bf6f-f91024964b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:45.862945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1108b-694c-431b-bf6f-f91024964b42 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:45.863481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1108b-694c-431b-bf6f-f91024964b42 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:45.863972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1108b-694c-431b-bf6f-f91024964b42 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:45.864817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1108b-694c-431b-bf6f-f91024964b42 None admin] RBAC: Adding query filter params (name=network) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:45.865341 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1108b-694c-431b-bf6f-f91024964b42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:45.866942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1108b-694c-431b-bf6f-f91024964b42 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:45.872225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 363/718] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 30 14:38:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:45.878060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6711bd1-80f9-4895-8e38-00c6f953f5ea None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:46.013676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6711bd1-80f9-4895-8e38-00c6f953f5ea None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:46.028610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:46.054261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6dc79-12a8-42e9-addd-847c222a8aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:46.055263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6dc79-12a8-42e9-addd-847c222a8aac None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:46.055740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6dc79-12a8-42e9-addd-847c222a8aac None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:46.056223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6dc79-12a8-42e9-addd-847c222a8aac None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:46.057067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6dc79-12a8-42e9-addd-847c222a8aac None admin] RBAC: Adding query filter params (type=network) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:46.058096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6dc79-12a8-42e9-addd-847c222a8aac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:46.059695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6dc79-12a8-42e9-addd-847c222a8aac None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:46.065041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 356/719] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 30 14:38:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:46.427447 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e236ee9-d29c-4014-8f23-13ea546a061d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:46.427856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e236ee9-d29c-4014-8f23-13ea546a061d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:46.428039 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e236ee9-d29c-4014-8f23-13ea546a061d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:46.429528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 364/720] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:46.435624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64b4af-d569-4c8f-86d7-fd991a5874ed None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:46.435914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64b4af-d569-4c8f-86d7-fd991a5874ed None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:46.436183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64b4af-d569-4c8f-86d7-fd991a5874ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:46.464224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-ce64b4af-d569-4c8f-86d7-fd991a5874ed None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:46.465429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ce64b4af-d569-4c8f-86d7-fd991a5874ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:46.551276 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 357/721] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:38:46.906095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e236ee9-d29c-4014-8f23-13ea546a061d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:46.988605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:47.036457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e236ee9-d29c-4014-8f23-13ea546a061d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:47.071281 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ecc9f-1278-4503-b0b3-f39c041a603a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:47.071918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ecc9f-1278-4503-b0b3-f39c041a603a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:47.072192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ecc9f-1278-4503-b0b3-f39c041a603a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:47.072461 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ecc9f-1278-4503-b0b3-f39c041a603a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:47.084208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ecc9f-1278-4503-b0b3-f39c041a603a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:47.089785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-079ecc9f-1278-4503-b0b3-f39c041a603a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:47.089785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:47.089785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:47.089785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:47.089785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:47.089785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:47.089785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:47.089785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:47.089785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:47.089785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:47.089785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:47.089785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:47.089785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:47.089785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:47.089785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:47.089785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:47.090768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:47.090768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:47.090768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:47.090768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:47.090768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:47.090768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:47.090768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:47.090768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:47.090768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:47.090768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:47.090768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:47.090768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:47.090768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:47.090768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:47.090768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:47.090768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:47.092648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:47.092648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:47.092648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:47.092648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:47.092648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:47.092648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:47.092648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:47.092648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:47.092648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:47.092648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:47.092648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:47.092648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:47.092648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 365/722] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:38:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:47.097097 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64b4af-d569-4c8f-86d7-fd991a5874ed None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:47.239125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64b4af-d569-4c8f-86d7-fd991a5874ed None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:47.245245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:47.247168 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf98bf-f98c-45cd-922b-5950065df5ac None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:47.247453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf98bf-f98c-45cd-922b-5950065df5ac None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:47.247723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf98bf-f98c-45cd-922b-5950065df5ac None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:47.249355 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 366/723] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:47.259486 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d20982-a3f6-4c08-8933-007dd63b9804 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:47.259764 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d20982-a3f6-4c08-8933-007dd63b9804 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:47.260037 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d20982-a3f6-4c08-8933-007dd63b9804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:47.283180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-67d20982-a3f6-4c08-8933-007dd63b9804 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:47.284433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-67d20982-a3f6-4c08-8933-007dd63b9804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:47.295999 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12054a8-05a1-4928-a2ef-b5762c067a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:47.296905 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12054a8-05a1-4928-a2ef-b5762c067a51 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:47.297403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12054a8-05a1-4928-a2ef-b5762c067a51 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:47.301396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12054a8-05a1-4928-a2ef-b5762c067a51 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:47.301396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12054a8-05a1-4928-a2ef-b5762c067a51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:47.301396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12054a8-05a1-4928-a2ef-b5762c067a51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:47.301396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12054a8-05a1-4928-a2ef-b5762c067a51 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:47.318636 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 358/724] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:47.336083 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12054a8-05a1-4928-a2ef-b5762c067a51 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:47.359876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 367/725] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:38:47.478568 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12054a8-05a1-4928-a2ef-b5762c067a51 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:47.521372 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb9cf2-f24c-44a1-b8a7-a2300601a9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:47.521929 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb9cf2-f24c-44a1-b8a7-a2300601a9a3 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:47.522196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb9cf2-f24c-44a1-b8a7-a2300601a9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:47.522469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb9cf2-f24c-44a1-b8a7-a2300601a9a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:47.539418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb9cf2-f24c-44a1-b8a7-a2300601a9a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:47.540856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb9cf2-f24c-44a1-b8a7-a2300601a9a3 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:47.559263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-78fb9cf2-f24c-44a1-b8a7-a2300601a9a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:47.559263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:47.559263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:47.559263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:47.559263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:47.559263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:47.559263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:47.559263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:47.559263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:47.559263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:47.559263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:47.559263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:47.559263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:47.559263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:47.559263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:47.559263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:47.560257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:47.561762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:47.561762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:47.561762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:47.561762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 359/726] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:47.566091 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d20982-a3f6-4c08-8933-007dd63b9804 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:47.628070 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:47.679565 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d20982-a3f6-4c08-8933-007dd63b9804 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:47.714745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809a698-d046-4802-b246-875d7472e509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:47.715373 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809a698-d046-4802-b246-875d7472e509 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:47.715651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809a698-d046-4802-b246-875d7472e509 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:47.715918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809a698-d046-4802-b246-875d7472e509 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:47.716557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809a698-d046-4802-b246-875d7472e509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:47.716854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809a698-d046-4802-b246-875d7472e509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:47.718214 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809a698-d046-4802-b246-875d7472e509 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:47.729792 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 368/727] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:47.735805 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb9cf2-f24c-44a1-b8a7-a2300601a9a3 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:47.766294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:47.785032 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809a698-d046-4802-b246-875d7472e509 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:47.909484 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb9cf2-f24c-44a1-b8a7-a2300601a9a3 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:47.915914 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809a698-d046-4802-b246-875d7472e509 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:47.931427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:47.952813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ea549-69ee-4dcd-ada5-d85d13d65271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:47.953893 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ea549-69ee-4dcd-ada5-d85d13d65271 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:47.954506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ea549-69ee-4dcd-ada5-d85d13d65271 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:47.954982 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ea549-69ee-4dcd-ada5-d85d13d65271 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:47.965565 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f32db-ed37-4ec6-8816-94f5979a62c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:47.966415 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f32db-ed37-4ec6-8816-94f5979a62c1 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:47.966869 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f32db-ed37-4ec6-8816-94f5979a62c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:47.967015 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ea549-69ee-4dcd-ada5-d85d13d65271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:47.967555 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f32db-ed37-4ec6-8816-94f5979a62c1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:47.972441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-f90ea549-69ee-4dcd-ada5-d85d13d65271 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:47.972441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:47.972441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:47.972441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:47.972441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:47.972441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:47.972441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:47.972441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:47.972441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:47.972441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:47.972441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:47.972441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:47.972441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:47.972441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:47.972441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:47.972441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:47.973581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:47.973581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:47.973581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:47.973581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:47.973581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:47.973581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:47.973581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:47.973581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:47.973581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:47.973581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:47.973581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:47.973581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:47.973581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:47.973581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:47.973581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:47.973581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:47.974627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:47.974627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:47.974627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:47.974627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:47.974627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:47.974627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:47.974627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:47.974627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:47.974627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:47.974627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:47.974627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:47.974627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:47.976169 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 369/728] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:47.977456 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca82f63-3704-4def-b5c6-14310dc3d6c3 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:47.977796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca82f63-3704-4def-b5c6-14310dc3d6c3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:47.978137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca82f63-3704-4def-b5c6-14310dc3d6c3 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:47.980270 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 370/729] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:47.980639 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f32db-ed37-4ec6-8816-94f5979a62c1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:47.981876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca82f63-3704-4def-b5c6-14310dc3d6c3 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:47.982611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f32db-ed37-4ec6-8816-94f5979a62c1 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:47.994835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-bf6f32db-ed37-4ec6-8816-94f5979a62c1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:38:47.994835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:47.994835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:47.994835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:47.994835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:47.994835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:47.994835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:47.994835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:47.994835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:47.994835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:47.994835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:47.994835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:47.994835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:47.994835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:47.994835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:47.994835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:47.996378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:47.997370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:47.997370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:47.997370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:47.997370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:47.997370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:47.997370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:47.997370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:47.997370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:47.997370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:38:47.997370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:47.999717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 360/730] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:38:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:48.001154 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324a438-ff98-492b-a99e-aeb3ee71ce7b None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:48.001617 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324a438-ff98-492b-a99e-aeb3ee71ce7b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:48.002067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324a438-ff98-492b-a99e-aeb3ee71ce7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:48.020659 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-3324a438-ff98-492b-a99e-aeb3ee71ce7b None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:48.022141 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-3324a438-ff98-492b-a99e-aeb3ee71ce7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:48.090217 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca82f63-3704-4def-b5c6-14310dc3d6c3 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:48.099142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 361/731] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:48.099142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324a438-ff98-492b-a99e-aeb3ee71ce7b None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:48.110644 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:48.127878 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122a38f-080b-4243-a28f-64738a653461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:48.128485 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122a38f-080b-4243-a28f-64738a653461 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:48.128815 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122a38f-080b-4243-a28f-64738a653461 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:48.129282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122a38f-080b-4243-a28f-64738a653461 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:48.129745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122a38f-080b-4243-a28f-64738a653461 None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:48.130051 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122a38f-080b-4243-a28f-64738a653461 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:48.131539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122a38f-080b-4243-a28f-64738a653461 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:48.140333 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 371/732] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:48.141653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7deded8-5559-4b1e-b283-d124f6eae26f None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:48.141941 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7deded8-5559-4b1e-b283-d124f6eae26f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:48.142213 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7deded8-5559-4b1e-b283-d124f6eae26f None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:48.144134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 372/733] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:48.145256 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7deded8-5559-4b1e-b283-d124f6eae26f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:48.178695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:48.210551 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324a438-ff98-492b-a99e-aeb3ee71ce7b None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:48.251636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7deded8-5559-4b1e-b283-d124f6eae26f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:48.255020 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e2d9a-e6f3-4e85-8c15-542b9ddcbd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:48.255932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e2d9a-e6f3-4e85-8c15-542b9ddcbd36 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:48.256425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e2d9a-e6f3-4e85-8c15-542b9ddcbd36 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:48.256898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e2d9a-e6f3-4e85-8c15-542b9ddcbd36 None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:48.257833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e2d9a-e6f3-4e85-8c15-542b9ddcbd36 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:48.258339 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e2d9a-e6f3-4e85-8c15-542b9ddcbd36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:48.260099 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e2d9a-e6f3-4e85-8c15-542b9ddcbd36 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:48.286201 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 362/734] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Aug 30 14:38:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:48.287404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0408d4b-2cb8-4366-aeb0-94acb955aeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:48.288068 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0408d4b-2cb8-4366-aeb0-94acb955aeb8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:48.288346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0408d4b-2cb8-4366-aeb0-94acb955aeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:48.288617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0408d4b-2cb8-4366-aeb0-94acb955aeb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:48.288840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbda907-c867-408f-9ea1-361710907e78 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:48.289322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbda907-c867-408f-9ea1-361710907e78 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:48.289790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbda907-c867-408f-9ea1-361710907e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:48.300008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0408d4b-2cb8-4366-aeb0-94acb955aeb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:48.301449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0408d4b-2cb8-4366-aeb0-94acb955aeb8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:48.310816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-cbbda907-c867-408f-9ea1-361710907e78 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:48.312398 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-cbbda907-c867-408f-9ea1-361710907e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:48.316221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-b0408d4b-2cb8-4366-aeb0-94acb955aeb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:48.316221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:48.316221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:48.316221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:48.316221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:48.316221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:48.316221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:48.316221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:48.316221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:48.316221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:48.316221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:48.316221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:48.316221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:48.316221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:48.316221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:48.316221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:48.317162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:48.318742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:48.318742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:48.318742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:48.318742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 373/735] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:48.318742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0408d4b-2cb8-4366-aeb0-94acb955aeb8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:48.384462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:48.390012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 363/736] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:48.391147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbda907-c867-408f-9ea1-361710907e78 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:48.407958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:48.433414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0408d4b-2cb8-4366-aeb0-94acb955aeb8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:48.479894 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1f638-5112-4069-b647-435678cf682a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:48.480576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1f638-5112-4069-b647-435678cf682a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:48.480854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1f638-5112-4069-b647-435678cf682a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:48.481126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1f638-5112-4069-b647-435678cf682a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:48.508163 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbda907-c867-408f-9ea1-361710907e78 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:48.551529 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddddffd-5985-4a5c-beaf-af33eb2e2897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:48.552134 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddddffd-5985-4a5c-beaf-af33eb2e2897 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:48.552408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddddffd-5985-4a5c-beaf-af33eb2e2897 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:48.552676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddddffd-5985-4a5c-beaf-af33eb2e2897 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:48.553313 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddddffd-5985-4a5c-beaf-af33eb2e2897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:48.553610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddddffd-5985-4a5c-beaf-af33eb2e2897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:48.554962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddddffd-5985-4a5c-beaf-af33eb2e2897 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:48.555819 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1f638-5112-4069-b647-435678cf682a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:48.557313 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1f638-5112-4069-b647-435678cf682a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:48.571238 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 364/737] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:48.572324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddddffd-5985-4a5c-beaf-af33eb2e2897 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:48.632200 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:48.643217 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:48.664660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 374/738] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:48.664660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32903bf-4d93-44ea-b7da-15a014ad27c6 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:48.664660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32903bf-4d93-44ea-b7da-15a014ad27c6 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:48.664660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32903bf-4d93-44ea-b7da-15a014ad27c6 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:48.664660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 375/739] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:48.664660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32903bf-4d93-44ea-b7da-15a014ad27c6 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:48.679326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddddffd-5985-4a5c-beaf-af33eb2e2897 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:48.712582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5eaa32-ba57-4b82-bc85-233314039c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:48.713182 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5eaa32-ba57-4b82-bc85-233314039c03 None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:48.714035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5eaa32-ba57-4b82-bc85-233314039c03 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:48.714292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5eaa32-ba57-4b82-bc85-233314039c03 None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:48.714927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5eaa32-ba57-4b82-bc85-233314039c03 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:48.716309 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5eaa32-ba57-4b82-bc85-233314039c03 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:48.769613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32903bf-4d93-44ea-b7da-15a014ad27c6 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:48.791619 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 365/740] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 30 14:38:48 2022] POST /identity/v3/services => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:38:48.792836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5eaa32-ba57-4b82-bc85-233314039c03 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:48.810726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0beb3c8-60e0-4e63-817f-f93ce570a7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:48.811421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0beb3c8-60e0-4e63-817f-f93ce570a7c1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:48.811684 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0beb3c8-60e0-4e63-817f-f93ce570a7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:48.811951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0beb3c8-60e0-4e63-817f-f93ce570a7c1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:48.820352 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0beb3c8-60e0-4e63-817f-f93ce570a7c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:48.821856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0beb3c8-60e0-4e63-817f-f93ce570a7c1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:48.835892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-a0beb3c8-60e0-4e63-817f-f93ce570a7c1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:38:48.835892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:48.835892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:48.835892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:48.835892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:48.835892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:48.835892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:48.835892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:48.835892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:48.835892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:48.835892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:48.835892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:48.835892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:48.835892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:48.835892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:48.835892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:48.837113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:48.838263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:48.838263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:48.838263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:48.838263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:48.838263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:48.838263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:48.838263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:48.838263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:48.838263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:38:48.838263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:48.843333 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 376/741] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 30 14:38:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:48.844947 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f35014-d4bd-4866-b941-005e773483c3 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:48.845255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f35014-d4bd-4866-b941-005e773483c3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:48.845528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f35014-d4bd-4866-b941-005e773483c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:48.869162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-51f35014-d4bd-4866-b941-005e773483c3 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:48.870539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-51f35014-d4bd-4866-b941-005e773483c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:48.908943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:48.919037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5eaa32-ba57-4b82-bc85-233314039c03 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:48.942485 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:48.945023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 377/742] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:38:48.946769 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f35014-d4bd-4866-b941-005e773483c3 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:48.960990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52134b-a4b4-40fb-8be8-54943eb2d7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:48.962132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52134b-a4b4-40fb-8be8-54943eb2d7bc None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:48.962132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52134b-a4b4-40fb-8be8-54943eb2d7bc None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:48.962301 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52134b-a4b4-40fb-8be8-54943eb2d7bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:48.976859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52134b-a4b4-40fb-8be8-54943eb2d7bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:48.978362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52134b-a4b4-40fb-8be8-54943eb2d7bc None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:48.995938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-9e52134b-a4b4-40fb-8be8-54943eb2d7bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:48.995938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:48.995938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:48.995938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:48.995938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:48.995938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:48.995938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:48.995938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:48.995938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:48.995938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:48.995938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:48.995938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:48.995938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:48.995938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:48.995938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:48.995938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:48.996895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:48.997780 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:48.997780 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:48.997780 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:49.000027 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 366/743] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:49.000964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52134b-a4b4-40fb-8be8-54943eb2d7bc None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:49.082372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f35014-d4bd-4866-b941-005e773483c3 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:49.117856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52134b-a4b4-40fb-8be8-54943eb2d7bc None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:49.126614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b46fb-3325-4458-b5d1-524f411a02bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:49.127283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b46fb-3325-4458-b5d1-524f411a02bf None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:49.127553 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b46fb-3325-4458-b5d1-524f411a02bf None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:49.127815 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b46fb-3325-4458-b5d1-524f411a02bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:49.141428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b46fb-3325-4458-b5d1-524f411a02bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:49.148289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-f95b46fb-3325-4458-b5d1-524f411a02bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:49.148289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:49.148289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:49.148289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:49.148289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:49.148289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:49.148289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:49.148289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:49.148289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:49.148289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:49.148289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:49.148289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:49.148289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:49.148289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:49.148289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:49.148289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:49.149244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:49.149244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:49.149244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:49.149244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:49.149244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:49.149244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:49.149244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:49.149244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:49.149244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:49.149244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:49.149244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:49.149244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:49.149244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:49.149244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:49.149244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:49.149244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:49.150337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:49.150337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:49.150337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:49.150337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:49.150337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:49.150337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:49.150337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:49.150337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:49.150337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:49.150337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:49.150337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:49.150337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:49.150337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 378/744] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:38:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:49.152180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b46fb-3325-4458-b5d1-524f411a02bf None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:49.159745 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720d533-fb6c-400f-b625-bd06769d8e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:49.160635 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720d533-fb6c-400f-b625-bd06769d8e0a None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:49.161112 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720d533-fb6c-400f-b625-bd06769d8e0a None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:49.161742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720d533-fb6c-400f-b625-bd06769d8e0a None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:49.162684 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720d533-fb6c-400f-b625-bd06769d8e0a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:49.163256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720d533-fb6c-400f-b625-bd06769d8e0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:49.165135 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720d533-fb6c-400f-b625-bd06769d8e0a None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:49.193386 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:49.193570 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 367/745] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Aug 30 14:38:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:49.193570 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720d533-fb6c-400f-b625-bd06769d8e0a None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:49.222674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:49.280577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b46fb-3325-4458-b5d1-524f411a02bf None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:49.302235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720d533-fb6c-400f-b625-bd06769d8e0a None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:49.318760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43201ffa-81cd-42d6-b8b7-2e374ba27f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:49.319416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43201ffa-81cd-42d6-b8b7-2e374ba27f53 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:49.319697 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43201ffa-81cd-42d6-b8b7-2e374ba27f53 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:49.319970 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43201ffa-81cd-42d6-b8b7-2e374ba27f53 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:49.320737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43201ffa-81cd-42d6-b8b7-2e374ba27f53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:49.321041 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43201ffa-81cd-42d6-b8b7-2e374ba27f53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:49.322594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43201ffa-81cd-42d6-b8b7-2e374ba27f53 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:49.337455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 379/746] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:49.338441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43201ffa-81cd-42d6-b8b7-2e374ba27f53 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:49.343685 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e64e3-fa61-40f3-ab45-9dc64b5c5cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:49.350623 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e64e3-fa61-40f3-ab45-9dc64b5c5cc2 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:49.350623 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e64e3-fa61-40f3-ab45-9dc64b5c5cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:49.350623 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e64e3-fa61-40f3-ab45-9dc64b5c5cc2 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:49.350623 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e64e3-fa61-40f3-ab45-9dc64b5c5cc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:49.350623 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e64e3-fa61-40f3-ab45-9dc64b5c5cc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:49.350623 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e64e3-fa61-40f3-ab45-9dc64b5c5cc2 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:49.360293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 368/747] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:49.360293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e64e3-fa61-40f3-ab45-9dc64b5c5cc2 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:49.437182 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:49.446957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43201ffa-81cd-42d6-b8b7-2e374ba27f53 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:49.455312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:49.465496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e64e3-fa61-40f3-ab45-9dc64b5c5cc2 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:49.484647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a03660-7f46-4823-8910-a98d08185667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:49.485250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a03660-7f46-4823-8910-a98d08185667 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:49.485538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a03660-7f46-4823-8910-a98d08185667 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:49.485790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a03660-7f46-4823-8910-a98d08185667 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:49.505730 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5824134-b6f1-48f1-9260-8239de72bdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:49.506332 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5824134-b6f1-48f1-9260-8239de72bdff None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:49.506606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5824134-b6f1-48f1-9260-8239de72bdff None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:49.506938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5824134-b6f1-48f1-9260-8239de72bdff None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:49.507624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5824134-b6f1-48f1-9260-8239de72bdff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:49.508970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5824134-b6f1-48f1-9260-8239de72bdff None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:49.553806 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a03660-7f46-4823-8910-a98d08185667 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:49.555647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a03660-7f46-4823-8910-a98d08185667 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:49.559142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 369/748] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 30 14:38:49 2022] POST /identity/v3/services => generated 240 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:38:49.559142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5824134-b6f1-48f1-9260-8239de72bdff None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:49.681012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:49.690436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5824134-b6f1-48f1-9260-8239de72bdff None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:49.698748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 380/749] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:49 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:38:49.700065 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a03660-7f46-4823-8910-a98d08185667 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:49.729134 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b8770-053f-4104-b7c1-6f2cd58ed462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:49.729769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b8770-053f-4104-b7c1-6f2cd58ed462 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:49.730040 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b8770-053f-4104-b7c1-6f2cd58ed462 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:49.730357 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b8770-053f-4104-b7c1-6f2cd58ed462 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:49.737879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b8770-053f-4104-b7c1-6f2cd58ed462 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:49.739341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b8770-053f-4104-b7c1-6f2cd58ed462 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:49.752545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-861b8770-053f-4104-b7c1-6f2cd58ed462 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:49.752545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:49.752545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:49.752545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:49.752545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:49.752545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:49.752545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:49.752545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:49.752545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:49.752545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:49.752545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:49.752545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:49.752545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:49.752545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:49.752545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:49.753511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:49.754691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:49.754691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:49.754691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:49.754691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:49.754691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:49.754691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:49.754691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:49.754691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:49.754691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 370/750] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:49.754691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b8770-053f-4104-b7c1-6f2cd58ed462 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:49.769561 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:49.828858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a03660-7f46-4823-8910-a98d08185667 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:49.866242 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b8770-053f-4104-b7c1-6f2cd58ed462 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:49.873389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f581e-d749-4cfb-ada4-69a39df66b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:49.874658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f581e-d749-4cfb-ada4-69a39df66b64 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:49.875130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f581e-d749-4cfb-ada4-69a39df66b64 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:49.875546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f581e-d749-4cfb-ada4-69a39df66b64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:49.890818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f581e-d749-4cfb-ada4-69a39df66b64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:49.892948 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f581e-d749-4cfb-ada4-69a39df66b64 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:49.901514 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2d003-3fa3-4813-984d-a1076ba39849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:49.902300 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2d003-3fa3-4813-984d-a1076ba39849 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:49.902419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2d003-3fa3-4813-984d-a1076ba39849 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:49.902674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2d003-3fa3-4813-984d-a1076ba39849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:49.911968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-8e9f581e-d749-4cfb-ada4-69a39df66b64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:49.911968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:49.911968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:49.911968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:49.911968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:49.911968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:49.911968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:49.911968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:49.911968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:49.911968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:49.911968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:49.911968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:49.911968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:49.911968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:49.911968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:49.911968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:49.913107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:49.914192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:49.914192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:49.914192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:49.915505 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2d003-3fa3-4813-984d-a1076ba39849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:49.916356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 381/751] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:49.917190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2d003-3fa3-4813-984d-a1076ba39849 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:49.917527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f581e-d749-4cfb-ada4-69a39df66b64 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:49.933289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-a7d2d003-3fa3-4813-984d-a1076ba39849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:49.933289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:49.933289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:49.933289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:49.933289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:49.933289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:49.933289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:49.933289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:49.933289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:49.933289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:49.933289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:49.933289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:49.933289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:49.933289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:49.933289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:49.933289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:49.934504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:49.935646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:49.935646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:49.935646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:49.937100 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 371/752] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:49.937978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2d003-3fa3-4813-984d-a1076ba39849 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:49.976250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:50.017398 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:50.039263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f581e-d749-4cfb-ada4-69a39df66b64 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:50.073459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2d003-3fa3-4813-984d-a1076ba39849 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:50.074943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f89564-999a-4793-b507-554633d8cac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:50.075689 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f89564-999a-4793-b507-554633d8cac7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.076067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f89564-999a-4793-b507-554633d8cac7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.076450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f89564-999a-4793-b507-554633d8cac7 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.077212 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f89564-999a-4793-b507-554633d8cac7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:50.077590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f89564-999a-4793-b507-554633d8cac7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:50.079114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f89564-999a-4793-b507-554633d8cac7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:50.092418 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 382/753] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:50.093437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f89564-999a-4793-b507-554633d8cac7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:50.115702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00612058-2e19-4548-ad90-347dd909a96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:50.116716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00612058-2e19-4548-ad90-347dd909a96f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.117256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00612058-2e19-4548-ad90-347dd909a96f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.117688 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00612058-2e19-4548-ad90-347dd909a96f None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.118625 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00612058-2e19-4548-ad90-347dd909a96f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:50.120350 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00612058-2e19-4548-ad90-347dd909a96f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:50.122115 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00612058-2e19-4548-ad90-347dd909a96f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:50.137666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 372/754] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:50.139037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00612058-2e19-4548-ad90-347dd909a96f None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:50.212456 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f89564-999a-4793-b507-554633d8cac7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:50.214331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:50.257774 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548aff9-f1e1-4f15-b61f-2dfd5a493344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:50.259124 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548aff9-f1e1-4f15-b61f-2dfd5a493344 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.259574 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548aff9-f1e1-4f15-b61f-2dfd5a493344 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.260110 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548aff9-f1e1-4f15-b61f-2dfd5a493344 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.264207 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548aff9-f1e1-4f15-b61f-2dfd5a493344 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:50.264207 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548aff9-f1e1-4f15-b61f-2dfd5a493344 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:50.265244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548aff9-f1e1-4f15-b61f-2dfd5a493344 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:50.281109 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 383/755] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:50.282145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548aff9-f1e1-4f15-b61f-2dfd5a493344 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:50.290549 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00612058-2e19-4548-ad90-347dd909a96f None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:50.316791 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:50.339461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de3133-21e1-49fd-a844-ebecf9fd5096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:50.340627 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de3133-21e1-49fd-a844-ebecf9fd5096 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.341082 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de3133-21e1-49fd-a844-ebecf9fd5096 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.341528 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de3133-21e1-49fd-a844-ebecf9fd5096 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.343048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de3133-21e1-49fd-a844-ebecf9fd5096 None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=bc23480d5aa94cad9e6599762ce01548) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:50.343675 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de3133-21e1-49fd-a844-ebecf9fd5096 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:50.347203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de3133-21e1-49fd-a844-ebecf9fd5096 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:50.355721 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 373/756] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:38:50 2022] GET /identity/v3/role_assignments?user.id=bc23480d5aa94cad9e6599762ce01548&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:50.356864 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de3133-21e1-49fd-a844-ebecf9fd5096 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:50.392923 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548aff9-f1e1-4f15-b61f-2dfd5a493344 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:50.465658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cd363-9076-4c08-be9a-103d900c6a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:50.466387 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cd363-9076-4c08-be9a-103d900c6a41 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.466667 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cd363-9076-4c08-be9a-103d900c6a41 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.466937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cd363-9076-4c08-be9a-103d900c6a41 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.548045 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de3133-21e1-49fd-a844-ebecf9fd5096 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:50.599816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9dfa2-5230-4e0f-ac5b-518ca080cbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:50.600464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9dfa2-5230-4e0f-ac5b-518ca080cbdd None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.600743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9dfa2-5230-4e0f-ac5b-518ca080cbdd None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.601012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9dfa2-5230-4e0f-ac5b-518ca080cbdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.612030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cd363-9076-4c08-be9a-103d900c6a41 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:50.613490 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cd363-9076-4c08-be9a-103d900c6a41 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:50.620262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9dfa2-5230-4e0f-ac5b-518ca080cbdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:50.621763 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9dfa2-5230-4e0f-ac5b-518ca080cbdd None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:50.625728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-903cd363-9076-4c08-be9a-103d900c6a41 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:38:50.625728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:50.625728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:50.625728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:50.625728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:50.625728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:50.625728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:50.625728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:50.625728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:50.625728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:50.625728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:50.625728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:50.625728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:50.625728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:50.625728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:50.625728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:50.626835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:50.628002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:50.628002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:50.628002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:50.628002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:50.628002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:50.628002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:50.628002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:50.628002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:50.628002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:38:50.628002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:50.630138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 384/757] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 30 14:38:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:50.640634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-80b9dfa2-5230-4e0f-ac5b-518ca080cbdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:50.640634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:50.640634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:50.640634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:50.640634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:50.640634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:50.640634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:50.640634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:50.640634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:50.640634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:50.640634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:50.640634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:50.640634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:50.640634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:50.640634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:50.641696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:50.642762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:50.642762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:50.642762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:50.642762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:50.642762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:50.642762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:50.642762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:50.642762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:50.642762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 374/758] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:50.647300 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cd363-9076-4c08-be9a-103d900c6a41 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:50.648458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9dfa2-5230-4e0f-ac5b-518ca080cbdd None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:50.701786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:50.708945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:50.768205 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9dfa2-5230-4e0f-ac5b-518ca080cbdd None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:50.828964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4ed7c-9100-4073-8ae3-23fe0b0d864a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:50.829492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4ed7c-9100-4073-8ae3-23fe0b0d864a None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.829765 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4ed7c-9100-4073-8ae3-23fe0b0d864a None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.830030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4ed7c-9100-4073-8ae3-23fe0b0d864a None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.830729 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4ed7c-9100-4073-8ae3-23fe0b0d864a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:50.831034 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4ed7c-9100-4073-8ae3-23fe0b0d864a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:50.832558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4ed7c-9100-4073-8ae3-23fe0b0d864a None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:50.851153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cd363-9076-4c08-be9a-103d900c6a41 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:50.851346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 375/759] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:50.851346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08f2cd-dfe4-4ef9-ba5e-45a61ff31cb5 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.851346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08f2cd-dfe4-4ef9-ba5e-45a61ff31cb5 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.851346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08f2cd-dfe4-4ef9-ba5e-45a61ff31cb5 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.851346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 376/760] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:50.853134 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08f2cd-dfe4-4ef9-ba5e-45a61ff31cb5 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:50.897595 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a34814-8064-40c8-8023-d1bf656fd9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:50.898347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a34814-8064-40c8-8023-d1bf656fd9f3 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.898723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a34814-8064-40c8-8023-d1bf656fd9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.899090 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a34814-8064-40c8-8023-d1bf656fd9f3 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.900662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a34814-8064-40c8-8023-d1bf656fd9f3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:50.901644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a34814-8064-40c8-8023-d1bf656fd9f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:50.904103 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a34814-8064-40c8-8023-d1bf656fd9f3 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:50.932447 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 385/761] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 30 14:38:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:50.932447 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a13de-eac6-4a73-b49c-d69297b0d1cf None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.932447 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a13de-eac6-4a73-b49c-d69297b0d1cf None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.932447 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a13de-eac6-4a73-b49c-d69297b0d1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.955684 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ac7a13de-eac6-4a73-b49c-d69297b0d1cf None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:50.957841 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ac7a13de-eac6-4a73-b49c-d69297b0d1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:50.970063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08f2cd-dfe4-4ef9-ba5e-45a61ff31cb5 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:50.971962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:51.008328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fbe98-03c0-41ec-9aac-f02434306dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:51.009194 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fbe98-03c0-41ec-9aac-f02434306dea None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:51.009611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fbe98-03c0-41ec-9aac-f02434306dea None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:51.010018 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fbe98-03c0-41ec-9aac-f02434306dea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:51.011879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fbe98-03c0-41ec-9aac-f02434306dea None admin] RBAC: Adding query filter params (role.id=244ad93ecd1949c9a6e8f74b135b43b4, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=912e9c732f194fa496a9111bbfab3efa) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:51.012334 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fbe98-03c0-41ec-9aac-f02434306dea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:51.014269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fbe98-03c0-41ec-9aac-f02434306dea None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:51.021816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 377/762] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:38:50 2022] GET /identity/v3/role_assignments?user.id=912e9c732f194fa496a9111bbfab3efa&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=244ad93ecd1949c9a6e8f74b135b43b4 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:51.022816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fbe98-03c0-41ec-9aac-f02434306dea None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:51.066130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 386/763] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:38:51.177335 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fbe98-03c0-41ec-9aac-f02434306dea None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:51.230862 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85080263-2da1-4136-a367-2bc95a7c0ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:51.231710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85080263-2da1-4136-a367-2bc95a7c0ae1 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:51.232096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85080263-2da1-4136-a367-2bc95a7c0ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:51.232468 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85080263-2da1-4136-a367-2bc95a7c0ae1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:51.248684 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85080263-2da1-4136-a367-2bc95a7c0ae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:51.250265 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85080263-2da1-4136-a367-2bc95a7c0ae1 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:51.265237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-85080263-2da1-4136-a367-2bc95a7c0ae1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:51.265237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:51.265237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:51.265237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:51.265237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:51.265237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:51.265237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:51.265237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:51.265237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:51.265237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:51.265237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:51.265237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:51.265237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:51.265237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:51.265237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:51.265237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:51.266394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:51.267498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:51.267498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:51.267498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:51.269490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 378/764] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:51.275281 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a13de-eac6-4a73-b49c-d69297b0d1cf None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:51.289218 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:51.381402 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a13de-eac6-4a73-b49c-d69297b0d1cf None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:51.398658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85080263-2da1-4136-a367-2bc95a7c0ae1 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:51.417999 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51250055-7e5a-4322-864e-eb476bf91e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:51.418330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51250055-7e5a-4322-864e-eb476bf91e88 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:51.418647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51250055-7e5a-4322-864e-eb476bf91e88 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:51.418920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51250055-7e5a-4322-864e-eb476bf91e88 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:51.419650 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51250055-7e5a-4322-864e-eb476bf91e88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:51.419954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51250055-7e5a-4322-864e-eb476bf91e88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:51.421317 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51250055-7e5a-4322-864e-eb476bf91e88 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:51.423314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:51.437301 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 387/765] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:51.439468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51250055-7e5a-4322-864e-eb476bf91e88 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:51.510177 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:51.517706 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85080263-2da1-4136-a367-2bc95a7c0ae1 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:51.543636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51250055-7e5a-4322-864e-eb476bf91e88 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:51.564764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f9c8b-af64-4ccf-aaec-0a7b742c1777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:51.565728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f9c8b-af64-4ccf-aaec-0a7b742c1777 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:51.566283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f9c8b-af64-4ccf-aaec-0a7b742c1777 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:51.566750 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f9c8b-af64-4ccf-aaec-0a7b742c1777 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:51.567549 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f9c8b-af64-4ccf-aaec-0a7b742c1777 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:51.569264 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f9c8b-af64-4ccf-aaec-0a7b742c1777 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-ac7f9c8b-af64-4ccf-aaec-0a7b742c1777 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:51.580003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:51.581012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:51.581012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:51.581012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:51.581012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:51.581012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:51.581012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:51.581012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:38:51.581012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:38:51.581012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:38:51.581012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:38:51.581012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:51.581012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:51.581012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:51.581012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:51.581012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:51.581012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:51.581831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:51.581831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:51.581831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:51.581831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:51.581831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:51.581831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:51.581831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:51.581831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:51.581831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:51.581831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:51.581831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:38:51.581831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:38:51.581831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:38:51.581831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:51.582658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980e8f2-a939-4090-bd28-97a0e7d0fbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:51.582658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980e8f2-a939-4090-bd28-97a0e7d0fbe4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:51.582658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980e8f2-a939-4090-bd28-97a0e7d0fbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:51.582658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980e8f2-a939-4090-bd28-97a0e7d0fbe4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:51.584745 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 379/766] 10.222.0.27 () {56 vars in 1129 bytes} [Tue Aug 30 14:38:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:38:51.590327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980e8f2-a939-4090-bd28-97a0e7d0fbe4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:51.590937 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f9c8b-af64-4ccf-aaec-0a7b742c1777 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:51.592059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980e8f2-a939-4090-bd28-97a0e7d0fbe4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:51.604900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-8980e8f2-a939-4090-bd28-97a0e7d0fbe4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:51.604900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:51.604900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:51.604900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:51.604900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:51.604900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:51.604900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:51.604900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:51.604900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:51.604900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:51.604900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:51.604900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:51.604900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:51.604900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:51.604900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:51.605977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:51.607035 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:51.607035 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:51.607035 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:51.607035 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:51.607035 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:51.607035 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:51.607035 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:51.607035 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:51.608802 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 388/767] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:38:51.614073 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980e8f2-a939-4090-bd28-97a0e7d0fbe4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:51.689256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:51.710702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f9c8b-af64-4ccf-aaec-0a7b742c1777 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:51.721943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980e8f2-a939-4090-bd28-97a0e7d0fbe4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:51.751489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:51.751976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823597a1-a837-4764-9d4a-c580432d7c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:51.753152 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823597a1-a837-4764-9d4a-c580432d7c72 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:51.753635 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823597a1-a837-4764-9d4a-c580432d7c72 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:51.754208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823597a1-a837-4764-9d4a-c580432d7c72 None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:51.755224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823597a1-a837-4764-9d4a-c580432d7c72 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:51.755857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823597a1-a837-4764-9d4a-c580432d7c72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:51.757536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda06b0c-3586-4ad6-b3a6-cf24ab39cfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:51.757949 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823597a1-a837-4764-9d4a-c580432d7c72 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:51.758230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda06b0c-3586-4ad6-b3a6-cf24ab39cfc6 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:51.758503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda06b0c-3586-4ad6-b3a6-cf24ab39cfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:51.758769 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda06b0c-3586-4ad6-b3a6-cf24ab39cfc6 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:51.759495 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda06b0c-3586-4ad6-b3a6-cf24ab39cfc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:51.759797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda06b0c-3586-4ad6-b3a6-cf24ab39cfc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:51.761243 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda06b0c-3586-4ad6-b3a6-cf24ab39cfc6 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:51.764488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 380/768] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Aug 30 14:38:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:51.770215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823597a1-a837-4764-9d4a-c580432d7c72 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:51.775114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 389/769] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:51.780422 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda06b0c-3586-4ad6-b3a6-cf24ab39cfc6 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:51.890155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda06b0c-3586-4ad6-b3a6-cf24ab39cfc6 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:51.892054 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823597a1-a837-4764-9d4a-c580432d7c72 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:51.927057 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af6888-4da8-41be-927d-0e6045fb5f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:51.927840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af6888-4da8-41be-927d-0e6045fb5f77 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:51.928214 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af6888-4da8-41be-927d-0e6045fb5f77 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:51.928510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af6888-4da8-41be-927d-0e6045fb5f77 None admin] PATH_INFO: `/v3/projects/1147574d59c7404baf76a24149684f6e/users/5b6e4c5bf2a54b28a55e5ae74b90ab4f/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:51.930506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af6888-4da8-41be-927d-0e6045fb5f77 None admin] RBAC: Authorizing `identity:create_grant(project_id=1147574d59c7404baf76a24149684f6e, user_id=5b6e4c5bf2a54b28a55e5ae74b90ab4f, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:51.932134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af6888-4da8-41be-927d-0e6045fb5f77 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:51.940124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a54e8-bb12-4384-bce7-004c3852c012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:51.940962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a54e8-bb12-4384-bce7-004c3852c012 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:51.941417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a54e8-bb12-4384-bce7-004c3852c012 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:51.941895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a54e8-bb12-4384-bce7-004c3852c012 None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:51.942722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a54e8-bb12-4384-bce7-004c3852c012 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:51.943222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a54e8-bb12-4384-bce7-004c3852c012 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:51.944776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a54e8-bb12-4384-bce7-004c3852c012 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:51.950430 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 381/770] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Aug 30 14:38:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:51.951753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffe6cc-2bad-4a52-a0d4-a7ad9b385449 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:51.952249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffe6cc-2bad-4a52-a0d4-a7ad9b385449 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:51.952683 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffe6cc-2bad-4a52-a0d4-a7ad9b385449 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:51.954328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 382/771] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:51.955741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffe6cc-2bad-4a52-a0d4-a7ad9b385449 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:51.968496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:52.003975 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 390/772] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:38:51 2022] PUT /identity/v3/projects/1147574d59c7404baf76a24149684f6e/users/5b6e4c5bf2a54b28a55e5ae74b90ab4f/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:38:52.009429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235b919-0d01-4a9b-9e93-8276db314791 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:52.009705 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235b919-0d01-4a9b-9e93-8276db314791 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:52.010463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235b919-0d01-4a9b-9e93-8276db314791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:52.052660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a235b919-0d01-4a9b-9e93-8276db314791 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:52.054271 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a235b919-0d01-4a9b-9e93-8276db314791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:52.114664 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffe6cc-2bad-4a52-a0d4-a7ad9b385449 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:52.153913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:52.180592 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e03434-b818-42de-ac15-37b72563df29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:52.181461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e03434-b818-42de-ac15-37b72563df29 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:52.181902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e03434-b818-42de-ac15-37b72563df29 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:52.182395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e03434-b818-42de-ac15-37b72563df29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:52.183247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e03434-b818-42de-ac15-37b72563df29 None admin] RBAC: Adding query filter params (interface=public, service_id=efe6c8e1008e4ef6903abd05f4a1b337, region_id=RegionOne) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:52.183657 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e03434-b818-42de-ac15-37b72563df29 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:52.185162 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e03434-b818-42de-ac15-37b72563df29 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:52.194096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 383/773] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Aug 30 14:38:51 2022] GET /identity/v3/endpoints?service_id=efe6c8e1008e4ef6903abd05f4a1b337&interface=public®ion_id=RegionOne => generated 194 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:52.203643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 391/774] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:38:52.204899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e03434-b818-42de-ac15-37b72563df29 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:52.290951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:52.319610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e03434-b818-42de-ac15-37b72563df29 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:52.372742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746371e-fb21-440e-9834-b67cae20e796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:52.373562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746371e-fb21-440e-9834-b67cae20e796 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:52.373940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746371e-fb21-440e-9834-b67cae20e796 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:52.374356 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746371e-fb21-440e-9834-b67cae20e796 None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:52.375064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746371e-fb21-440e-9834-b67cae20e796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:52.376619 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746371e-fb21-440e-9834-b67cae20e796 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:52.388336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 384/775] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 30 14:38:52 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:52.606484 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235b919-0d01-4a9b-9e93-8276db314791 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:52.693559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0f242-b253-41b9-b627-cfc00b64a230 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:52.694191 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0f242-b253-41b9-b627-cfc00b64a230 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:52.694658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0f242-b253-41b9-b627-cfc00b64a230 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:52.696587 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 385/776] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:38:52.702493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c435ba-e12b-478b-814a-978fd768ffd1 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:52.703027 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c435ba-e12b-478b-814a-978fd768ffd1 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:52.703539 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c435ba-e12b-478b-814a-978fd768ffd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:52.726644 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-48c435ba-e12b-478b-814a-978fd768ffd1 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:52.728057 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-48c435ba-e12b-478b-814a-978fd768ffd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:52.733837 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235b919-0d01-4a9b-9e93-8276db314791 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:52.774541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0a5d3-8138-4844-bb51-23adcff13db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:52.775827 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0a5d3-8138-4844-bb51-23adcff13db9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:52.776569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0a5d3-8138-4844-bb51-23adcff13db9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:52.777062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0a5d3-8138-4844-bb51-23adcff13db9 None admin] PATH_INFO: `/v3/services/network` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:52.778196 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c0a5d3-8138-4844-bb51-23adcff13db9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:52.779793 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c0a5d3-8138-4844-bb51-23adcff13db9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-92c0a5d3-8138-4844-bb51-23adcff13db9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:52.791030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:52.792186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:52.792186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:52.792186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:52.792186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:52.792186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:52.792186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:52.792186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:38:52.792186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:38:52.792186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:38:52.792186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:38:52.792186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:52.792186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:52.792186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:52.792186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:52.792186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:52.792186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:52.793071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:52.793071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:52.793071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:52.793071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:52.793071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:52.793071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:52.793071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:52.793071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:52.793071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:52.793071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:52.793071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:38:52.793071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:38:52.793071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:38:52.793071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:52.796088 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 392/777] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:52 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:52.802685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0a5d3-8138-4844-bb51-23adcff13db9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:52.865454 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:52.867163 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 386/778] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:38:52.931723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0a5d3-8138-4844-bb51-23adcff13db9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:52.970585 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f08bc-bd54-4a13-aa84-f8bc89a3e080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:52.971218 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f08bc-bd54-4a13-aa84-f8bc89a3e080 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:52.971500 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f08bc-bd54-4a13-aa84-f8bc89a3e080 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:52.971776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f08bc-bd54-4a13-aa84-f8bc89a3e080 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:52.972407 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f08bc-bd54-4a13-aa84-f8bc89a3e080 None admin] RBAC: Adding query filter params (name=network) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:52.972711 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f08bc-bd54-4a13-aa84-f8bc89a3e080 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:52.974078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f08bc-bd54-4a13-aa84-f8bc89a3e080 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:52.978921 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 393/779] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 30 14:38:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:52.984208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c435ba-e12b-478b-814a-978fd768ffd1 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:53.059457 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:53.089271 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c435ba-e12b-478b-814a-978fd768ffd1 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:53.126187 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cf4f9-3306-4c50-a899-a4f08e97708f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:53.126718 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cf4f9-3306-4c50-a899-a4f08e97708f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:53.126995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cf4f9-3306-4c50-a899-a4f08e97708f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:53.127289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cf4f9-3306-4c50-a899-a4f08e97708f None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:53.127923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cf4f9-3306-4c50-a899-a4f08e97708f None admin] RBAC: Adding query filter params (type=network) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:53.128397 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cf4f9-3306-4c50-a899-a4f08e97708f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:53.129772 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cf4f9-3306-4c50-a899-a4f08e97708f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:53.135355 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 387/780] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 30 14:38:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:53.144200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f08bc-bd54-4a13-aa84-f8bc89a3e080 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:53.263833 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f08bc-bd54-4a13-aa84-f8bc89a3e080 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:53.279447 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:53.310822 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095bd4e-0438-4ea0-bba3-51bc1ad2ab09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:53.311043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cf4f9-3306-4c50-a899-a4f08e97708f None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:53.311612 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095bd4e-0438-4ea0-bba3-51bc1ad2ab09 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:53.311876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095bd4e-0438-4ea0-bba3-51bc1ad2ab09 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:53.312145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095bd4e-0438-4ea0-bba3-51bc1ad2ab09 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:53.312932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095bd4e-0438-4ea0-bba3-51bc1ad2ab09 None admin] RBAC: Adding query filter params (interface=public, service_id=80102c9ad4fa41c7a72055b99efee9f3, region_id=RegionOne) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:53.313264 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095bd4e-0438-4ea0-bba3-51bc1ad2ab09 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:53.314750 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7095bd4e-0438-4ea0-bba3-51bc1ad2ab09 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:53.330661 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 394/781] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Aug 30 14:38:53 2022] GET /identity/v3/endpoints?service_id=80102c9ad4fa41c7a72055b99efee9f3&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:53.336211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095bd4e-0438-4ea0-bba3-51bc1ad2ab09 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:53.455990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cf4f9-3306-4c50-a899-a4f08e97708f None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:53.461075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095bd4e-0438-4ea0-bba3-51bc1ad2ab09 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:53.487669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:53.492828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdf24c-badb-4662-ba8c-7086617a640f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:53.493591 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdf24c-badb-4662-ba8c-7086617a640f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:53.493965 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdf24c-badb-4662-ba8c-7086617a640f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:53.494240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdf24c-badb-4662-ba8c-7086617a640f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:53.505099 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fdf24c-badb-4662-ba8c-7086617a640f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:53.506554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ef808-cb72-4014-bb5a-4f12a6c0669e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:53.507190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ef808-cb72-4014-bb5a-4f12a6c0669e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:53.507470 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ef808-cb72-4014-bb5a-4f12a6c0669e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:53.510440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ef808-cb72-4014-bb5a-4f12a6c0669e None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:53.510440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ef808-cb72-4014-bb5a-4f12a6c0669e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:53.510440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ef808-cb72-4014-bb5a-4f12a6c0669e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:53.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-f3fdf24c-badb-4662-ba8c-7086617a640f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:53.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:53.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:53.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:53.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:53.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:53.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:53.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:53.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:53.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:53.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:53.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:53.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:53.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:53.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:53.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:53.512111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:53.512111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:53.512111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:53.512111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:53.512111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:53.512111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:53.512111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:53.512111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:53.512111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:53.512111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:53.512111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:53.512111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:53.512111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:53.512111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:53.512111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:53.512111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:53.514118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:53.514118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:53.514118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:53.514118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:53.514118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:53.514118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:53.514118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:53.514118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:53.514118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:53.514118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:53.514118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:53.514118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:53.514118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 388/782] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:38:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:38:53.519129 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdf24c-badb-4662-ba8c-7086617a640f None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:53.519312 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 395/783] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 30 14:38:53 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:53.615216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e1a61-ba9f-45bb-ad19-61e9e187adda None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:53.615543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e1a61-ba9f-45bb-ad19-61e9e187adda None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:53.615762 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e1a61-ba9f-45bb-ad19-61e9e187adda None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:53.617441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 396/784] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:53.628684 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f1471-5b15-43a5-bc46-ccd6cd638624 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:53.628986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f1471-5b15-43a5-bc46-ccd6cd638624 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:53.629245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f1471-5b15-43a5-bc46-ccd6cd638624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:53.659967 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4d5f1471-5b15-43a5-bc46-ccd6cd638624 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:53.661186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4d5f1471-5b15-43a5-bc46-ccd6cd638624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:53.691048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdf24c-badb-4662-ba8c-7086617a640f None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:53.741170 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:53.750391 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5a036-5d3e-4cbb-9c84-afffe4a9efc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:53.751000 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5a036-5d3e-4cbb-9c84-afffe4a9efc2 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:53.751312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5a036-5d3e-4cbb-9c84-afffe4a9efc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:53.756729 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5a036-5d3e-4cbb-9c84-afffe4a9efc2 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:53.756729 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5a036-5d3e-4cbb-9c84-afffe4a9efc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:53.756729 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5a036-5d3e-4cbb-9c84-afffe4a9efc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:53.756729 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5a036-5d3e-4cbb-9c84-afffe4a9efc2 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:53.764546 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 389/785] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:53.770272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5a036-5d3e-4cbb-9c84-afffe4a9efc2 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:53.794197 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 397/786] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:38:53.833223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:53.879880 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5a036-5d3e-4cbb-9c84-afffe4a9efc2 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:53.917768 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c2a95-be25-4cab-9252-541b15b42e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:53.918457 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c2a95-be25-4cab-9252-541b15b42e19 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:53.918738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c2a95-be25-4cab-9252-541b15b42e19 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:53.919007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c2a95-be25-4cab-9252-541b15b42e19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:53.930772 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c2a95-be25-4cab-9252-541b15b42e19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:53.932371 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c2a95-be25-4cab-9252-541b15b42e19 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:53.948294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-a03c2a95-be25-4cab-9252-541b15b42e19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:53.948294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:53.948294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:53.948294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:53.948294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:53.948294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:53.948294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:53.948294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:53.948294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:53.948294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:53.948294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:53.948294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:53.948294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:53.948294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:53.948294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:53.948294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:53.949236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:53.951225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:53.951225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:53.951225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:53.951225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 390/787] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:53.957149 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f1471-5b15-43a5-bc46-ccd6cd638624 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:54.094642 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f1471-5b15-43a5-bc46-ccd6cd638624 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:54.132735 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe02a57-b1ea-4925-8617-16ca4aeb09ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:54.133344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe02a57-b1ea-4925-8617-16ca4aeb09ea None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:54.133623 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe02a57-b1ea-4925-8617-16ca4aeb09ea None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:54.133894 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe02a57-b1ea-4925-8617-16ca4aeb09ea None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:54.134543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe02a57-b1ea-4925-8617-16ca4aeb09ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:54.134843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe02a57-b1ea-4925-8617-16ca4aeb09ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:54.136839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe02a57-b1ea-4925-8617-16ca4aeb09ea None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:54.154119 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 398/788] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:54.159896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c2a95-be25-4cab-9252-541b15b42e19 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:54.197660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe02a57-b1ea-4925-8617-16ca4aeb09ea None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:54.228525 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:54.326109 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe02a57-b1ea-4925-8617-16ca4aeb09ea None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:54.362438 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d644f-79bf-462d-a87c-e21b3815ed3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:54.362978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d644f-79bf-462d-a87c-e21b3815ed3c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:54.363286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d644f-79bf-462d-a87c-e21b3815ed3c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:54.363560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d644f-79bf-462d-a87c-e21b3815ed3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:54.374482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d644f-79bf-462d-a87c-e21b3815ed3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:54.379237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-996d644f-79bf-462d-a87c-e21b3815ed3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:54.379237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:54.379237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:54.379237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:54.379237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:54.379237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:54.379237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:54.379237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:54.379237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:54.379237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:54.379237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:54.379237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:54.379237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:54.379237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:54.379237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:54.379237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:54.380191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:54.380191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:54.380191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:54.380191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:54.380191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:54.380191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:54.380191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:54.380191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:54.380191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:54.380191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:54.380191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:54.380191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:54.380191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:54.380191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:54.380191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:54.380191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:54.381050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:54.381050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:54.381050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:54.381050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:54.381050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:54.381050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:54.381050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:54.381050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:54.381050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:54.381050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:54.381050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:38:54.381050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:54.381050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 399/789] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:38:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:38:54.382343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c724f47-4272-4875-a037-648affe4de0c None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:54.382637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c724f47-4272-4875-a037-648affe4de0c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:54.382907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c724f47-4272-4875-a037-648affe4de0c None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:54.388454 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 400/790] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:54.388454 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c724f47-4272-4875-a037-648affe4de0c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:54.392938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c2a95-be25-4cab-9252-541b15b42e19 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:54.405209 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:54.443451 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eda16e-ceeb-4436-9d22-b8a1ffa4c62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:54.444726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eda16e-ceeb-4436-9d22-b8a1ffa4c62a None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:54.445602 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eda16e-ceeb-4436-9d22-b8a1ffa4c62a None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:54.446173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eda16e-ceeb-4436-9d22-b8a1ffa4c62a None admin] PATH_INFO: `/v3/users/swift` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:54.457079 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eda16e-ceeb-4436-9d22-b8a1ffa4c62a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:54.459576 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eda16e-ceeb-4436-9d22-b8a1ffa4c62a None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:54.478359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-a6eda16e-ceeb-4436-9d22-b8a1ffa4c62a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:38:54.478359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:54.478359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:54.478359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:54.478359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:54.478359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:54.478359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:54.478359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:54.478359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:54.478359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:54.478359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:54.478359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:54.478359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:54.478359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:54.478359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:54.478359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:54.480193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:54.481133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:54.481133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:54.481133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:54.481133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:54.481133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:54.481133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:54.481133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:54.481133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:54.481133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:38:54.481133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:54.484530 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 391/791] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:38:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:54.486435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:54.487793 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e205b11-4538-4380-a20f-bc36e205db76 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:54.488305 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e205b11-4538-4380-a20f-bc36e205db76 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:54.489031 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e205b11-4538-4380-a20f-bc36e205db76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:54.503191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c724f47-4272-4875-a037-648affe4de0c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:54.514931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-2e205b11-4538-4380-a20f-bc36e205db76 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:54.516832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-2e205b11-4538-4380-a20f-bc36e205db76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:54.544792 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f623c-dc14-47b2-a508-5f7d2154a1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:54.545452 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f623c-dc14-47b2-a508-5f7d2154a1cc None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:54.545730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f623c-dc14-47b2-a508-5f7d2154a1cc None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:54.546000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f623c-dc14-47b2-a508-5f7d2154a1cc None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:54.546710 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f623c-dc14-47b2-a508-5f7d2154a1cc None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:54.547015 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f623c-dc14-47b2-a508-5f7d2154a1cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:54.548415 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f623c-dc14-47b2-a508-5f7d2154a1cc None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:54.556929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 401/792] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:38:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:54.558307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f623c-dc14-47b2-a508-5f7d2154a1cc None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:54.619353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 392/793] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:38:54.620168 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e205b11-4538-4380-a20f-bc36e205db76 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:54.704328 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f623c-dc14-47b2-a508-5f7d2154a1cc None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:54.766887 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee038c-c32d-470d-8b1e-c27d212164c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:54.767526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee038c-c32d-470d-8b1e-c27d212164c6 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:54.767845 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee038c-c32d-470d-8b1e-c27d212164c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:54.768121 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee038c-c32d-470d-8b1e-c27d212164c6 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:54.768852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee038c-c32d-470d-8b1e-c27d212164c6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:54.769156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee038c-c32d-470d-8b1e-c27d212164c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:54.770586 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee038c-c32d-470d-8b1e-c27d212164c6 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:54.796473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:54.801269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 402/794] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Aug 30 14:38:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:54.803954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e205b11-4538-4380-a20f-bc36e205db76 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:54.807584 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee038c-c32d-470d-8b1e-c27d212164c6 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:54.822095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:54.843237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfd16b-9bbd-4cdd-9a2e-c6a3dea8ae3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:54.844160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfd16b-9bbd-4cdd-9a2e-c6a3dea8ae3c None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:54.844922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfd16b-9bbd-4cdd-9a2e-c6a3dea8ae3c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:54.845408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfd16b-9bbd-4cdd-9a2e-c6a3dea8ae3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:54.856399 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfd16b-9bbd-4cdd-9a2e-c6a3dea8ae3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:54.858174 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfd16b-9bbd-4cdd-9a2e-c6a3dea8ae3c None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:54.874212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-6cbfd16b-9bbd-4cdd-9a2e-c6a3dea8ae3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:54.874212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:54.874212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:54.874212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:54.874212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:54.874212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:54.874212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:54.874212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:54.874212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:54.874212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:54.874212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:54.874212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:54.874212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:54.874212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:54.874212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:54.874212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:54.875184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:54.876063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:54.876063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:54.876063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:54.878106 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 393/795] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:54.879454 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b046219-b407-462a-926b-5fcb1e08571f None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:54.879866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b046219-b407-462a-926b-5fcb1e08571f None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:54.880282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b046219-b407-462a-926b-5fcb1e08571f None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:54.881887 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 394/796] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:54.887226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b046219-b407-462a-926b-5fcb1e08571f None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:54.935780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee038c-c32d-470d-8b1e-c27d212164c6 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:54.974347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483a45b-ba03-4b5c-90f8-c58fa2bfa5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:54.974995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483a45b-ba03-4b5c-90f8-c58fa2bfa5d5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:54.975308 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483a45b-ba03-4b5c-90f8-c58fa2bfa5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:54.976137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483a45b-ba03-4b5c-90f8-c58fa2bfa5d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:54.989198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483a45b-ba03-4b5c-90f8-c58fa2bfa5d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:54.990651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483a45b-ba03-4b5c-90f8-c58fa2bfa5d5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:55.005897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-e483a45b-ba03-4b5c-90f8-c58fa2bfa5d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:55.005897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:55.005897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:55.005897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:55.005897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:55.005897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:55.005897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:55.005897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:55.005897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:55.005897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:55.005897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:55.005897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:55.005897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:55.005897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:55.005897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:55.005897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:55.006851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:55.007783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:55.007783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:55.007783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:55.007783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 403/797] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:55.008510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273c274-ebcb-4f3b-8fc9-10d653042b86 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:55.008510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273c274-ebcb-4f3b-8fc9-10d653042b86 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:55.008909 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273c274-ebcb-4f3b-8fc9-10d653042b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:55.013449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b046219-b407-462a-926b-5fcb1e08571f None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:55.026885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c273c274-ebcb-4f3b-8fc9-10d653042b86 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:55.028095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c273c274-ebcb-4f3b-8fc9-10d653042b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:55.050336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e885e1-4350-4f71-9666-85cb8774b352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:55.050939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e885e1-4350-4f71-9666-85cb8774b352 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:55.051251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e885e1-4350-4f71-9666-85cb8774b352 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:55.051527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e885e1-4350-4f71-9666-85cb8774b352 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:55.052164 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e885e1-4350-4f71-9666-85cb8774b352 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:55.052473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e885e1-4350-4f71-9666-85cb8774b352 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:55.053828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e885e1-4350-4f71-9666-85cb8774b352 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:55.065749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 395/798] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:55.066428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e885e1-4350-4f71-9666-85cb8774b352 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:55.080199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:55.094009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:55.109411 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 404/799] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:38:55.116004 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273c274-ebcb-4f3b-8fc9-10d653042b86 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:55.177848 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e885e1-4350-4f71-9666-85cb8774b352 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:55.214661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cd8c4-ec5b-47ca-9b07-94c3ab4255e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:55.215767 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cd8c4-ec5b-47ca-9b07-94c3ab4255e9 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:55.216309 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cd8c4-ec5b-47ca-9b07-94c3ab4255e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:55.216803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cd8c4-ec5b-47ca-9b07-94c3ab4255e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:55.225485 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273c274-ebcb-4f3b-8fc9-10d653042b86 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:55.228448 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0cd8c4-ec5b-47ca-9b07-94c3ab4255e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:55.239948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-7f0cd8c4-ec5b-47ca-9b07-94c3ab4255e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:55.239948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:55.239948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:55.239948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:55.239948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:55.239948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:55.239948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:55.239948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:55.239948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:55.239948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:55.239948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:55.239948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:55.239948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:55.239948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:55.239948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:55.239948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:55.240914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:55.240914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:55.240914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:55.240914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:55.240914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:55.240914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:55.240914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:55.240914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:55.240914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:55.240914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:55.240914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:55.240914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:55.240914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:55.240914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:55.240914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:55.240914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:55.241761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:55.241761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:55.241761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:55.241761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:55.241761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:55.241761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:55.241761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:55.241761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:55.241761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:55.241761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:55.241761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:55.241761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:55.245132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 396/800] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:38:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:55.246257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cd8c4-ec5b-47ca-9b07-94c3ab4255e9 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:55.268108 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f93f5-27c4-4edc-81f0-ee45fc686a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:55.268831 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f93f5-27c4-4edc-81f0-ee45fc686a9a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:55.269115 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f93f5-27c4-4edc-81f0-ee45fc686a9a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:55.269388 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f93f5-27c4-4edc-81f0-ee45fc686a9a None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:55.270285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f93f5-27c4-4edc-81f0-ee45fc686a9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:55.270459 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f93f5-27c4-4edc-81f0-ee45fc686a9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:55.272014 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f93f5-27c4-4edc-81f0-ee45fc686a9a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:55.285299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 405/801] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:55.286049 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f93f5-27c4-4edc-81f0-ee45fc686a9a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:55.328444 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:55.371273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:55.392879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cd8c4-ec5b-47ca-9b07-94c3ab4255e9 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:55.430353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a84363-782e-426c-b684-99276d2cff3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:55.430975 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f93f5-27c4-4edc-81f0-ee45fc686a9a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:55.431071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a84363-782e-426c-b684-99276d2cff3a None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:55.431168 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a84363-782e-426c-b684-99276d2cff3a None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:55.431442 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a84363-782e-426c-b684-99276d2cff3a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:55.439858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a84363-782e-426c-b684-99276d2cff3a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:55.441281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a84363-782e-426c-b684-99276d2cff3a None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:55.452917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-c0a84363-782e-426c-b684-99276d2cff3a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:38:55.452917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:55.452917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:55.452917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:55.452917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:55.452917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:55.452917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:55.452917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:55.452917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:55.452917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:55.452917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:55.452917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:55.452917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:55.452917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:55.452917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:55.452917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:55.453858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:55.454804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:55.454804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:55.454804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:55.454804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:55.454804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:55.454804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:55.454804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:55.454804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:55.454804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:38:55.454804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:55.454804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 397/802] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 30 14:38:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:55.454804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a84363-782e-426c-b684-99276d2cff3a None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:55.468763 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee85fd2-360f-4fc5-ae92-4472c808594b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:55.469364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee85fd2-360f-4fc5-ae92-4472c808594b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:55.469636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee85fd2-360f-4fc5-ae92-4472c808594b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:55.469901 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee85fd2-360f-4fc5-ae92-4472c808594b None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:55.470561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee85fd2-360f-4fc5-ae92-4472c808594b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:55.470859 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee85fd2-360f-4fc5-ae92-4472c808594b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:55.472245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee85fd2-360f-4fc5-ae92-4472c808594b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:55.481302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 406/803] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:55.481970 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee85fd2-360f-4fc5-ae92-4472c808594b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:55.569412 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a84363-782e-426c-b684-99276d2cff3a None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:55.583726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee85fd2-360f-4fc5-ae92-4472c808594b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:55.595964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:55.600828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:55.609201 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40c193-2fbd-443f-a205-e4b3b5a17859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:55.609832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40c193-2fbd-443f-a205-e4b3b5a17859 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:55.610142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40c193-2fbd-443f-a205-e4b3b5a17859 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:55.612840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40c193-2fbd-443f-a205-e4b3b5a17859 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:55.617386 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40c193-2fbd-443f-a205-e4b3b5a17859 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:55.618913 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40c193-2fbd-443f-a205-e4b3b5a17859 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:55.622929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae49d7-264b-46e6-adc4-f8387a82d609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:55.623584 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae49d7-264b-46e6-adc4-f8387a82d609 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:55.623862 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae49d7-264b-46e6-adc4-f8387a82d609 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:55.624187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae49d7-264b-46e6-adc4-f8387a82d609 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:55.624770 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae49d7-264b-46e6-adc4-f8387a82d609 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:55.626434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae49d7-264b-46e6-adc4-f8387a82d609 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:55.630898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-3e40c193-2fbd-443f-a205-e4b3b5a17859 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:55.630898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:55.630898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:55.630898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:55.630898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:55.630898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:55.630898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:55.630898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:55.630898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:55.630898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:55.630898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:55.630898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:55.630898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:55.630898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:55.630898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:55.631940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:55.637896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:55.637896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:55.637896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:55.637896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:55.637896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:55.637896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:55.637896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:55.637896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:55.637896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 398/804] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:55.637896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40c193-2fbd-443f-a205-e4b3b5a17859 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-25ae49d7-264b-46e6-adc4-f8387a82d609 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:55.638432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:55.647285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:55.647285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:55.647285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:55.647285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:55.647285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:55.647285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:55.647285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:38:55.647285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:38:55.647285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:38:55.647285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:38:55.647285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:55.647285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:55.647285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:55.647285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:55.647285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:55.647285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:55.648269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:55.648269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:55.648269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:55.648269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:55.648269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:55.648269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:55.648269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:55.648269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:55.648269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:55.648269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:55.648269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:38:55.648269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:38:55.648269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:38:55.648269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:55.648269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 407/805] 10.222.0.27 () {56 vars in 1129 bytes} [Tue Aug 30 14:38:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:55.649057 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae49d7-264b-46e6-adc4-f8387a82d609 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:55.744507 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae49d7-264b-46e6-adc4-f8387a82d609 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:55.749024 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40c193-2fbd-443f-a205-e4b3b5a17859 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:55.781626 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b7e2f-7d5b-47fc-8754-ed76a550f266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:55.782260 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b7e2f-7d5b-47fc-8754-ed76a550f266 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:55.782542 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b7e2f-7d5b-47fc-8754-ed76a550f266 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:55.782809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b7e2f-7d5b-47fc-8754-ed76a550f266 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:55.783161 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829cc3c7-7eac-46e3-83e3-f02c6bcac933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:55.783758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829cc3c7-7eac-46e3-83e3-f02c6bcac933 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:55.784033 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829cc3c7-7eac-46e3-83e3-f02c6bcac933 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:55.784298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829cc3c7-7eac-46e3-83e3-f02c6bcac933 None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:55.785013 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829cc3c7-7eac-46e3-83e3-f02c6bcac933 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:55.785313 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829cc3c7-7eac-46e3-83e3-f02c6bcac933 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:55.786756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829cc3c7-7eac-46e3-83e3-f02c6bcac933 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:55.794257 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b7e2f-7d5b-47fc-8754-ed76a550f266 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:55.795754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b7e2f-7d5b-47fc-8754-ed76a550f266 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:55.806677 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 399/806] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Aug 30 14:38:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:55.807539 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829cc3c7-7eac-46e3-83e3-f02c6bcac933 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:55.814995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-8a3b7e2f-7d5b-47fc-8754-ed76a550f266 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:55.814995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:55.814995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:55.814995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:55.814995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:55.814995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:55.814995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:55.814995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:55.814995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:55.814995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:55.814995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:55.814995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:55.814995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:55.814995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:55.814995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:55.814995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:55.816005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:55.816942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:55.816942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:55.816942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:55.817114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 408/807] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:55.822422 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b7e2f-7d5b-47fc-8754-ed76a550f266 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:55.846452 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:55.854769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:55.904919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829cc3c7-7eac-46e3-83e3-f02c6bcac933 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:55.921442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b7e2f-7d5b-47fc-8754-ed76a550f266 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:55.938212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c723aa9-2bdc-426b-a99b-2946446d4b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:55.938827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c723aa9-2bdc-426b-a99b-2946446d4b80 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:55.939078 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c723aa9-2bdc-426b-a99b-2946446d4b80 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:55.939384 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c723aa9-2bdc-426b-a99b-2946446d4b80 None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:55.940067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c723aa9-2bdc-426b-a99b-2946446d4b80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:55.940367 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c723aa9-2bdc-426b-a99b-2946446d4b80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:55.941765 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c723aa9-2bdc-426b-a99b-2946446d4b80 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:55.954007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 400/808] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:55.954792 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c723aa9-2bdc-426b-a99b-2946446d4b80 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:55.955034 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254115d-363d-4b7b-85dd-e9e5af008e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:55.955654 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254115d-363d-4b7b-85dd-e9e5af008e4c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:55.955931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254115d-363d-4b7b-85dd-e9e5af008e4c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:55.956196 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254115d-363d-4b7b-85dd-e9e5af008e4c None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:55.956815 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254115d-363d-4b7b-85dd-e9e5af008e4c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:55.958602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254115d-363d-4b7b-85dd-e9e5af008e4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:55.958602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254115d-363d-4b7b-85dd-e9e5af008e4c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:55.963703 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 409/809] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Aug 30 14:38:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:55.964759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaec758-1b32-4959-ba62-c2207db86ff3 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:55.965035 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaec758-1b32-4959-ba62-c2207db86ff3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:55.965302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaec758-1b32-4959-ba62-c2207db86ff3 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:55.966480 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 410/810] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:55.967221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaec758-1b32-4959-ba62-c2207db86ff3 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:56.056750 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:56.059083 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c723aa9-2bdc-426b-a99b-2946446d4b80 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:56.066440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:56.067019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaec758-1b32-4959-ba62-c2207db86ff3 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:56.093376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1d11e-408e-4df0-b8f6-611ca3f6bdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:56.094059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1d11e-408e-4df0-b8f6-611ca3f6bdfa None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:56.094341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1d11e-408e-4df0-b8f6-611ca3f6bdfa None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:56.094614 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1d11e-408e-4df0-b8f6-611ca3f6bdfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:56.100633 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330efcf-0fc0-4edf-b45b-17be15581b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:56.101233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330efcf-0fc0-4edf-b45b-17be15581b10 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:56.101505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330efcf-0fc0-4edf-b45b-17be15581b10 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:56.101772 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330efcf-0fc0-4edf-b45b-17be15581b10 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:56.102405 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0330efcf-0fc0-4edf-b45b-17be15581b10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:56.102703 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0330efcf-0fc0-4edf-b45b-17be15581b10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:56.104163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0330efcf-0fc0-4edf-b45b-17be15581b10 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:56.104659 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1d11e-408e-4df0-b8f6-611ca3f6bdfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:56.106089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1d11e-408e-4df0-b8f6-611ca3f6bdfa None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:56.115081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 411/811] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:56.115921 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330efcf-0fc0-4edf-b45b-17be15581b10 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:56.125341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-2fa1d11e-408e-4df0-b8f6-611ca3f6bdfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:56.125341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:56.125341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:56.125341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:56.125341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:56.125341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:56.125341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:56.125341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:56.125341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:56.125341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:56.125341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:56.125341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:56.125341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:56.125341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:56.125341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:56.125341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:56.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:56.132304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:56.132304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:56.132304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:56.132304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 401/812] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:56.132304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1d11e-408e-4df0-b8f6-611ca3f6bdfa None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:56.219377 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330efcf-0fc0-4edf-b45b-17be15581b10 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:56.226221 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1d11e-408e-4df0-b8f6-611ca3f6bdfa None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:56.252573 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fefee0-59e5-45a7-8015-0ffe2018b694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:56.253220 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fefee0-59e5-45a7-8015-0ffe2018b694 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:56.253496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fefee0-59e5-45a7-8015-0ffe2018b694 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:56.253770 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fefee0-59e5-45a7-8015-0ffe2018b694 None admin] PATH_INFO: `/v3/projects/1147574d59c7404baf76a24149684f6e/users/bc23480d5aa94cad9e6599762ce01548/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:56.255661 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fefee0-59e5-45a7-8015-0ffe2018b694 None admin] RBAC: Authorizing `identity:create_grant(project_id=1147574d59c7404baf76a24149684f6e, user_id=bc23480d5aa94cad9e6599762ce01548, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:56.257093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fefee0-59e5-45a7-8015-0ffe2018b694 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:56.262000 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b015737-c262-452b-94bd-554ea2b9a3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:56.262603 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b015737-c262-452b-94bd-554ea2b9a3b0 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:56.262879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b015737-c262-452b-94bd-554ea2b9a3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:56.263170 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b015737-c262-452b-94bd-554ea2b9a3b0 None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:56.263811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b015737-c262-452b-94bd-554ea2b9a3b0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:56.264110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b015737-c262-452b-94bd-554ea2b9a3b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:56.265462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b015737-c262-452b-94bd-554ea2b9a3b0 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:56.270508 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 402/813] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Aug 30 14:38:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:56.271691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350153db-bcba-40c4-94d2-c21b651f55e2 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:56.272012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350153db-bcba-40c4-94d2-c21b651f55e2 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:56.272285 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350153db-bcba-40c4-94d2-c21b651f55e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:56.292473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-350153db-bcba-40c4-94d2-c21b651f55e2 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:56.292473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-350153db-bcba-40c4-94d2-c21b651f55e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:56.307708 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:56.333878 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 412/814] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:38:56 2022] PUT /identity/v3/projects/1147574d59c7404baf76a24149684f6e/users/bc23480d5aa94cad9e6599762ce01548/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:38:56.334693 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fefee0-59e5-45a7-8015-0ffe2018b694 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:56.359515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:56.360438 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 403/815] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:38:56.361443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350153db-bcba-40c4-94d2-c21b651f55e2 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:56.485961 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fefee0-59e5-45a7-8015-0ffe2018b694 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:56.532920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cdcd4-15dd-4a21-839b-bbdda6da0370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:56.533459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350153db-bcba-40c4-94d2-c21b651f55e2 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:56.533883 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cdcd4-15dd-4a21-839b-bbdda6da0370 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:56.533883 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cdcd4-15dd-4a21-839b-bbdda6da0370 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:56.534092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cdcd4-15dd-4a21-839b-bbdda6da0370 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:56.543780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cdcd4-15dd-4a21-839b-bbdda6da0370 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:56.545846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cdcd4-15dd-4a21-839b-bbdda6da0370 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:56.562389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-116cdcd4-15dd-4a21-839b-bbdda6da0370 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:38:56.562389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:56.562389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:56.562389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:56.562389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:56.562389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:56.562389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:56.562389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:56.562389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:56.562389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:56.562389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:56.562389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:56.562389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:56.562389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:56.562389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:56.562389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:56.563371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:56.564311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:56.564311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:56.564311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:56.564311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:56.564311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:56.564311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:56.564311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:56.564311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:56.564311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:38:56.564311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:56.564311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 413/816] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 30 14:38:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:56.564925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cdcd4-15dd-4a21-839b-bbdda6da0370 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:56.579038 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78f513-01f6-4025-9c00-a99b2d221eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:56.579683 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78f513-01f6-4025-9c00-a99b2d221eff None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:56.579963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78f513-01f6-4025-9c00-a99b2d221eff None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:56.580254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78f513-01f6-4025-9c00-a99b2d221eff None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:56.581165 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78f513-01f6-4025-9c00-a99b2d221eff None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:56.581474 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78f513-01f6-4025-9c00-a99b2d221eff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:56.583160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78f513-01f6-4025-9c00-a99b2d221eff None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:56.597870 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 404/817] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:56.599241 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78f513-01f6-4025-9c00-a99b2d221eff None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:56.620141 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:56.649266 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:56.684014 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cdcd4-15dd-4a21-839b-bbdda6da0370 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:56.719686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78f513-01f6-4025-9c00-a99b2d221eff None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:56.723167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773928b-40ac-4687-bcba-216eac53fb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:56.724001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773928b-40ac-4687-bcba-216eac53fb02 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:56.724505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773928b-40ac-4687-bcba-216eac53fb02 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:56.724984 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773928b-40ac-4687-bcba-216eac53fb02 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:56.725877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2773928b-40ac-4687-bcba-216eac53fb02 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:56.727671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2773928b-40ac-4687-bcba-216eac53fb02 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:56.754369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523142e7-1376-4349-9d29-184c7407b9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:56.754975 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523142e7-1376-4349-9d29-184c7407b9a6 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:56.755300 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523142e7-1376-4349-9d29-184c7407b9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:56.755558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523142e7-1376-4349-9d29-184c7407b9a6 None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:56.756289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523142e7-1376-4349-9d29-184c7407b9a6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:56.756596 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523142e7-1376-4349-9d29-184c7407b9a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:56.758021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523142e7-1376-4349-9d29-184c7407b9a6 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:56.775974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 405/818] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 30 14:38:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:56.776786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523142e7-1376-4349-9d29-184c7407b9a6 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:56.837824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 414/819] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:38:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:38:56.838655 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773928b-40ac-4687-bcba-216eac53fb02 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:56.862790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:56.898149 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523142e7-1376-4349-9d29-184c7407b9a6 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:56.945440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb71c9-b717-4fe5-95ff-2e6bfc08cff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:56.946349 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb71c9-b717-4fe5-95ff-2e6bfc08cff8 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:56.947233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb71c9-b717-4fe5-95ff-2e6bfc08cff8 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:56.947844 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb71c9-b717-4fe5-95ff-2e6bfc08cff8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:56.955606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb71c9-b717-4fe5-95ff-2e6bfc08cff8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:56.957507 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb71c9-b717-4fe5-95ff-2e6bfc08cff8 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:56.970606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-dddb71c9-b717-4fe5-95ff-2e6bfc08cff8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:56.970606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:56.970606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:56.970606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:56.970606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:56.970606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:56.970606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:56.970606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:56.970606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:56.970606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:56.970606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:56.970606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:56.970606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:56.970606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:56.970606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:56.971724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:56.972618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:56.972618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:56.972618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:56.972618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:56.972618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:56.972618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:56.972618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:56.972618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:56.975590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 406/820] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:56.976672 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb71c9-b717-4fe5-95ff-2e6bfc08cff8 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:56.993125 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773928b-40ac-4687-bcba-216eac53fb02 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:57.030505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:57.043469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873ef90-8393-46ce-9105-6346dfbb77a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:57.056463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873ef90-8393-46ce-9105-6346dfbb77a9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:57.056463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873ef90-8393-46ce-9105-6346dfbb77a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:57.056463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873ef90-8393-46ce-9105-6346dfbb77a9 None admin] PATH_INFO: `/v3/services/network` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:57.056463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873ef90-8393-46ce-9105-6346dfbb77a9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:57.056463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873ef90-8393-46ce-9105-6346dfbb77a9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-e873ef90-8393-46ce-9105-6346dfbb77a9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:57.058713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:57.059997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:57.059997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:57.059997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:57.059997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:57.059997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:57.059997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:57.059997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:38:57.059997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:38:57.059997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:38:57.059997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:38:57.059997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:57.059997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:57.059997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:57.059997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:57.059997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:57.059997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:57.061860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:57.061860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:57.061860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:57.061860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:57.061860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:57.061860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:57.061860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:57.061860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:57.061860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:57.061860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:57.061860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:38:57.061860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:38:57.061860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:38:57.061860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:57.061860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 415/821] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:56 2022] GET /identity/v3/services/network => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:57.062664 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873ef90-8393-46ce-9105-6346dfbb77a9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:57.111702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb71c9-b717-4fe5-95ff-2e6bfc08cff8 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:57.139244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:57.149484 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ae252-8dbc-4005-a37c-5383bff75d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:57.151094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ae252-8dbc-4005-a37c-5383bff75d7f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:57.151604 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ae252-8dbc-4005-a37c-5383bff75d7f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:57.152071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ae252-8dbc-4005-a37c-5383bff75d7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:57.200023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873ef90-8393-46ce-9105-6346dfbb77a9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:57.237389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39268fa9-5acf-4d04-a7f3-d1817ee0966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:57.238405 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39268fa9-5acf-4d04-a7f3-d1817ee0966b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:57.238887 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39268fa9-5acf-4d04-a7f3-d1817ee0966b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:57.239392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39268fa9-5acf-4d04-a7f3-d1817ee0966b None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:57.240336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39268fa9-5acf-4d04-a7f3-d1817ee0966b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:57.240852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39268fa9-5acf-4d04-a7f3-d1817ee0966b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:57.242478 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39268fa9-5acf-4d04-a7f3-d1817ee0966b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:57.243678 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ae252-8dbc-4005-a37c-5383bff75d7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:57.246070 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ae252-8dbc-4005-a37c-5383bff75d7f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:57.256120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 416/822] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:57.257280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39268fa9-5acf-4d04-a7f3-d1817ee0966b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:57.313025 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:57.366153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39268fa9-5acf-4d04-a7f3-d1817ee0966b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:57.385672 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 407/823] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:38:57.385672 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ae252-8dbc-4005-a37c-5383bff75d7f None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:57.406852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24e353-a0c9-4e55-b6b8-57831d51e11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:57.407823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24e353-a0c9-4e55-b6b8-57831d51e11b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:57.408309 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24e353-a0c9-4e55-b6b8-57831d51e11b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:57.408780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24e353-a0c9-4e55-b6b8-57831d51e11b None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:57.409633 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24e353-a0c9-4e55-b6b8-57831d51e11b None admin] RBAC: Adding query filter params (name=network) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:57.410144 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24e353-a0c9-4e55-b6b8-57831d51e11b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:57.411739 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24e353-a0c9-4e55-b6b8-57831d51e11b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:57.415133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:57.417750 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 417/824] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 30 14:38:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:57.418907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24e353-a0c9-4e55-b6b8-57831d51e11b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:57.509245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ae252-8dbc-4005-a37c-5383bff75d7f None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:57.538616 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24e353-a0c9-4e55-b6b8-57831d51e11b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:57.546678 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0cc22-1d56-4436-bd66-1d621708f0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:57.547963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0cc22-1d56-4436-bd66-1d621708f0d6 None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:57.548487 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0cc22-1d56-4436-bd66-1d621708f0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:57.549027 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0cc22-1d56-4436-bd66-1d621708f0d6 None admin] PATH_INFO: `/v3/projects/1147574d59c7404baf76a24149684f6e/users/912e9c732f194fa496a9111bbfab3efa/roles/244ad93ecd1949c9a6e8f74b135b43b4` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:57.555146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:57.564742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0cc22-1d56-4436-bd66-1d621708f0d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=1147574d59c7404baf76a24149684f6e, user_id=912e9c732f194fa496a9111bbfab3efa, role_id=244ad93ecd1949c9a6e8f74b135b43b4)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:57.566420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0cc22-1d56-4436-bd66-1d621708f0d6 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:57.579242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f717a2-d440-4861-94c9-1569900c4196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:57.583787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f717a2-d440-4861-94c9-1569900c4196 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:57.583787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f717a2-d440-4861-94c9-1569900c4196 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:57.583787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f717a2-d440-4861-94c9-1569900c4196 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:57.591290 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f717a2-d440-4861-94c9-1569900c4196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:57.592730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f717a2-d440-4861-94c9-1569900c4196 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:57.606545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-51f717a2-d440-4861-94c9-1569900c4196 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:57.606545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:57.606545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:57.606545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:57.606545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:57.606545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:57.606545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:57.606545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:57.606545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:57.606545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:57.606545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:57.606545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:57.606545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:57.606545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:57.606545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:57.606545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:57.607483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:57.608430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:57.608430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:57.608430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:57.613330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 418/825] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:57.613330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f717a2-d440-4861-94c9-1569900c4196 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:57.665705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 408/826] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:38:57 2022] PUT /identity/v3/projects/1147574d59c7404baf76a24149684f6e/users/912e9c732f194fa496a9111bbfab3efa/roles/244ad93ecd1949c9a6e8f74b135b43b4 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:38:57.671324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0cc22-1d56-4436-bd66-1d621708f0d6 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:57.735365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f717a2-d440-4861-94c9-1569900c4196 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:57.788641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706cfa1-815b-4492-8ff8-a1ff985f63ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:57.789574 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706cfa1-815b-4492-8ff8-a1ff985f63ea None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:57.790005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706cfa1-815b-4492-8ff8-a1ff985f63ea None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:57.790421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706cfa1-815b-4492-8ff8-a1ff985f63ea None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:57.791943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706cfa1-815b-4492-8ff8-a1ff985f63ea None admin] RBAC: Adding query filter params (type=network) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:57.792520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706cfa1-815b-4492-8ff8-a1ff985f63ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:57.794156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706cfa1-815b-4492-8ff8-a1ff985f63ea None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:57.805701 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 419/827] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 30 14:38:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:57.810611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:57.811483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706cfa1-815b-4492-8ff8-a1ff985f63ea None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:57.832822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0cc22-1d56-4436-bd66-1d621708f0d6 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:57.875379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:57.880183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d3499-3802-4d63-9f5f-e1ec6d3470ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:57.880789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d3499-3802-4d63-9f5f-e1ec6d3470ed None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:57.881061 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d3499-3802-4d63-9f5f-e1ec6d3470ed None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:57.881328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d3499-3802-4d63-9f5f-e1ec6d3470ed None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:57.881979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d3499-3802-4d63-9f5f-e1ec6d3470ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:57.882275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d3499-3802-4d63-9f5f-e1ec6d3470ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:57.883674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d3499-3802-4d63-9f5f-e1ec6d3470ed None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:57.896649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 409/828] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:57.903373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d3499-3802-4d63-9f5f-e1ec6d3470ed None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:57.957049 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706cfa1-815b-4492-8ff8-a1ff985f63ea None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:57.998084 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecd37a-8d18-420d-ae36-f564d008464e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:57.998707 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecd37a-8d18-420d-ae36-f564d008464e None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:57.998987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecd37a-8d18-420d-ae36-f564d008464e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:58.001734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecd37a-8d18-420d-ae36-f564d008464e None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:58.001734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ecd37a-8d18-420d-ae36-f564d008464e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:58.001734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ecd37a-8d18-420d-ae36-f564d008464e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:58.008968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d3499-3802-4d63-9f5f-e1ec6d3470ed None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:58.044651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73cb8c-68f6-469e-8c9d-274c58a6dc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:58.045275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73cb8c-68f6-469e-8c9d-274c58a6dc6a None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:58.045550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73cb8c-68f6-469e-8c9d-274c58a6dc6a None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:58.045816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73cb8c-68f6-469e-8c9d-274c58a6dc6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:58.052989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73cb8c-68f6-469e-8c9d-274c58a6dc6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:58.054465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73cb8c-68f6-469e-8c9d-274c58a6dc6a None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:58.067029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-6e73cb8c-68f6-469e-8c9d-274c58a6dc6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:58.067029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:58.067029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:58.067029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:58.067029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:58.067029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:58.067029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:58.067029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:58.067029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:58.067029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:58.067029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:58.067029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:58.067029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:38:58.067029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:38:58.067029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:58.069231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:58.070450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:38:58.070450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:38:58.070450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:38:58.070450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:38:58.070450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:38:58.070450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:38:58.070450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:38:58.070450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:58.070450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 410/829] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:38:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:58.070450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:58.074312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73cb8c-68f6-469e-8c9d-274c58a6dc6a None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:58.089310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 420/830] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:38:57 2022] POST /identity/v3/endpoints => generated 337 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:38:58.183322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73cb8c-68f6-469e-8c9d-274c58a6dc6a None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:58.225901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a4462-a031-4ad0-b31c-a386c5cf9bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:58.226510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a4462-a031-4ad0-b31c-a386c5cf9bbb None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:58.226782 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a4462-a031-4ad0-b31c-a386c5cf9bbb None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:58.227051 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a4462-a031-4ad0-b31c-a386c5cf9bbb None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:58.227772 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a4462-a031-4ad0-b31c-a386c5cf9bbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:58.228070 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a4462-a031-4ad0-b31c-a386c5cf9bbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:58.230150 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a4462-a031-4ad0-b31c-a386c5cf9bbb None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:58.244577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 411/831] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:38:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:58.250490 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecd37a-8d18-420d-ae36-f564d008464e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:58.380281 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:58.397561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecd37a-8d18-420d-ae36-f564d008464e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:58.428270 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5187ae3-91b8-4840-9fac-59655b738c85 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:58.428610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5187ae3-91b8-4840-9fac-59655b738c85 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:58.428815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5187ae3-91b8-4840-9fac-59655b738c85 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:58.430142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 412/832] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:38:58.435601 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041465e4-950b-431f-9bfe-426bb0bf863e None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:58.435887 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041465e4-950b-431f-9bfe-426bb0bf863e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:58.436179 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041465e4-950b-431f-9bfe-426bb0bf863e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:58.438268 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32353215-ca91-4258-92cb-2ef1e2d4c7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:58.438868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32353215-ca91-4258-92cb-2ef1e2d4c7c4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:58.439163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32353215-ca91-4258-92cb-2ef1e2d4c7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:58.439492 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32353215-ca91-4258-92cb-2ef1e2d4c7c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:58.440220 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32353215-ca91-4258-92cb-2ef1e2d4c7c4 None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=5b6e4c5bf2a54b28a55e5ae74b90ab4f) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:58.440523 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32353215-ca91-4258-92cb-2ef1e2d4c7c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:58.441979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32353215-ca91-4258-92cb-2ef1e2d4c7c4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:58.457874 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 421/833] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:38:58 2022] GET /identity/v3/role_assignments?user.id=5b6e4c5bf2a54b28a55e5ae74b90ab4f&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:58.462810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-041465e4-950b-431f-9bfe-426bb0bf863e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:58.464654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-041465e4-950b-431f-9bfe-426bb0bf863e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:58.525325 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:58.549304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 413/834] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:38:58.823665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32353215-ca91-4258-92cb-2ef1e2d4c7c4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:58.929409 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32353215-ca91-4258-92cb-2ef1e2d4c7c4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:58.965666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896bf5c-ba01-4460-9adb-772e7b979a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:58.966303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896bf5c-ba01-4460-9adb-772e7b979a1b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:58.966585 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896bf5c-ba01-4460-9adb-772e7b979a1b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:58.966854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896bf5c-ba01-4460-9adb-772e7b979a1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:58.977541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896bf5c-ba01-4460-9adb-772e7b979a1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:38:58.982591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-5896bf5c-ba01-4460-9adb-772e7b979a1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:58.982591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:58.982591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:58.982591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:58.982591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:58.982591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:58.982591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:58.982591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:58.982591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:58.982591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:58.982591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:58.982591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:58.982591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:38:58.982591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:38:58.982591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:38:58.982591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:38:58.983583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:38:58.983583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:38:58.983583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:38:58.983583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:38:58.983583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:58.983583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:58.983583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:58.983583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:58.983583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:58.983583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:58.983583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:58.983583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:58.983583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:58.983583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:58.983583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:58.983583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:58.984457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:58.984457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:58.984457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:58.984457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:58.984457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:38:58.984457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:38:58.984457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:38:58.984457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:38:58.984457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:38:58.984457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:38:58.984457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:38:58.984457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:58.984457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 422/835] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:38:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:58.989977 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041465e4-950b-431f-9bfe-426bb0bf863e None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:59.018740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7131cf-3d74-4616-b9e9-e51f85138637 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:59.019419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7131cf-3d74-4616-b9e9-e51f85138637 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:59.019939 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7131cf-3d74-4616-b9e9-e51f85138637 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:59.022307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 423/836] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:59.027790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa58c74-0118-4f58-8308-df8626679661 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:59.028317 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa58c74-0118-4f58-8308-df8626679661 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:59.028794 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa58c74-0118-4f58-8308-df8626679661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:59.047717 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1aa58c74-0118-4f58-8308-df8626679661 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:59.049254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1aa58c74-0118-4f58-8308-df8626679661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:59.079571 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:59.107936 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041465e4-950b-431f-9bfe-426bb0bf863e None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:59.135616 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 424/837] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:38:59.147830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997965d1-1c1c-40b7-9050-7a5378af87ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:59.148662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997965d1-1c1c-40b7-9050-7a5378af87ef None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:59.149138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997965d1-1c1c-40b7-9050-7a5378af87ef None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:59.149868 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997965d1-1c1c-40b7-9050-7a5378af87ef None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:59.151087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997965d1-1c1c-40b7-9050-7a5378af87ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:59.151490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997965d1-1c1c-40b7-9050-7a5378af87ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:59.152861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997965d1-1c1c-40b7-9050-7a5378af87ef None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:59.161732 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 414/838] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:38:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:59.167809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa58c74-0118-4f58-8308-df8626679661 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:59.268839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa58c74-0118-4f58-8308-df8626679661 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:59.303001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d8db2-acab-4503-9abd-418debbfaba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:59.303659 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d8db2-acab-4503-9abd-418debbfaba2 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:59.303935 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d8db2-acab-4503-9abd-418debbfaba2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:59.304211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d8db2-acab-4503-9abd-418debbfaba2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:59.314381 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d8db2-acab-4503-9abd-418debbfaba2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:59.315867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d8db2-acab-4503-9abd-418debbfaba2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:59.323296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:59.337643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-554d8db2-acab-4503-9abd-418debbfaba2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:59.337643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:59.337643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:59.337643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:59.337643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:59.337643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:59.337643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:59.337643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:59.337643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:59.337643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:59.337643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:59.337643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:59.337643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:38:59.337643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:38:59.337643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:38:59.337643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:59.339244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:38:59.340135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:38:59.340135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:38:59.340135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:59.340135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 425/839] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:38:59.340369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997965d1-1c1c-40b7-9050-7a5378af87ef None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:59.402555 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:59.440237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d8db2-acab-4503-9abd-418debbfaba2 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:59.445643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997965d1-1c1c-40b7-9050-7a5378af87ef None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:59.490051 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf4a48-3360-4290-a587-1e51267505ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:59.491747 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf4a48-3360-4290-a587-1e51267505ca None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:59.492214 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf4a48-3360-4290-a587-1e51267505ca None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:59.492683 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf4a48-3360-4290-a587-1e51267505ca None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:59.493774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf4a48-3360-4290-a587-1e51267505ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:59.495284 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf4a48-3360-4290-a587-1e51267505ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:59.497504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf4a48-3360-4290-a587-1e51267505ca None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:59.516363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 415/840] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:59.520927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf4a48-3360-4290-a587-1e51267505ca None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:59.554825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d8db2-acab-4503-9abd-418debbfaba2 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:59.591598 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1715f-c08a-4ce6-b714-3d42720c82b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:59.592606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1715f-c08a-4ce6-b714-3d42720c82b2 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:59.593095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1715f-c08a-4ce6-b714-3d42720c82b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:59.593574 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1715f-c08a-4ce6-b714-3d42720c82b2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:59.594392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1715f-c08a-4ce6-b714-3d42720c82b2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:59.596101 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1715f-c08a-4ce6-b714-3d42720c82b2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-64d1715f-c08a-4ce6-b714-3d42720c82b2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:59.606370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:59.607508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:59.607508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:59.607508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:59.607508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:59.607508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:59.607508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:59.607508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:38:59.607508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:38:59.607508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:38:59.607508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:38:59.607508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:59.607508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:59.607508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:59.607508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:59.607508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:59.607508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:59.608435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:59.608435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:59.608435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:59.608435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:59.608435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:59.608435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:59.608435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:59.608435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:59.608435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:59.608435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:59.608435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:38:59.608435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:38:59.608435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:38:59.608435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:38:59.611336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 426/841] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:38:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:38:59.616660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1715f-c08a-4ce6-b714-3d42720c82b2 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:59.637063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf4a48-3360-4290-a587-1e51267505ca None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:59.640133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:59.673459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443cfa6-03c4-4cce-ad6b-b784adff650e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:59.674105 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443cfa6-03c4-4cce-ad6b-b784adff650e None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:59.674426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443cfa6-03c4-4cce-ad6b-b784adff650e None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:59.674636 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443cfa6-03c4-4cce-ad6b-b784adff650e None admin] PATH_INFO: `/v3/users/swift` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:59.682728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443cfa6-03c4-4cce-ad6b-b784adff650e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:59.684224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443cfa6-03c4-4cce-ad6b-b784adff650e None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:59.686346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:59.696563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-1443cfa6-03c4-4cce-ad6b-b784adff650e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:38:59.696563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:38:59.696563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:38:59.696563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:38:59.696563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:38:59.696563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:38:59.696563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:38:59.696563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:38:59.696563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:38:59.696563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:38:59.696563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:38:59.696563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:38:59.696563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:38:59.696563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:38:59.696563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:38:59.696563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:38:59.698428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:38:59.699410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:38:59.699410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:38:59.699410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:38:59.699410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:38:59.699410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:38:59.699410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:38:59.699410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:38:59.699410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:38:59.699410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:38:59.699410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:38:59.703588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 416/842] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:38:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:38:59.704497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443cfa6-03c4-4cce-ad6b-b784adff650e None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:59.739679 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1715f-c08a-4ce6-b714-3d42720c82b2 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:59.778999 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c0b69-630b-4444-9a77-c3f7f6d983eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:59.780012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c0b69-630b-4444-9a77-c3f7f6d983eb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:59.780500 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c0b69-630b-4444-9a77-c3f7f6d983eb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:59.780993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c0b69-630b-4444-9a77-c3f7f6d983eb None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:59.781860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c0b69-630b-4444-9a77-c3f7f6d983eb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:59.782380 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c0b69-630b-4444-9a77-c3f7f6d983eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:59.784382 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c0b69-630b-4444-9a77-c3f7f6d983eb None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:59.795048 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 427/843] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:38:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:38:59.796186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c0b69-630b-4444-9a77-c3f7f6d983eb None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:59.809772 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443cfa6-03c4-4cce-ad6b-b784adff650e None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:59.846144 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c00152-7691-4927-a9fa-fb2b064f403b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:59.846711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c00152-7691-4927-a9fa-fb2b064f403b None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:59.846944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c00152-7691-4927-a9fa-fb2b064f403b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:59.847247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c00152-7691-4927-a9fa-fb2b064f403b None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:59.847981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c00152-7691-4927-a9fa-fb2b064f403b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:59.848286 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c00152-7691-4927-a9fa-fb2b064f403b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:59.849776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c00152-7691-4927-a9fa-fb2b064f403b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:59.873445 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 417/844] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Aug 30 14:38:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:59.874493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c00152-7691-4927-a9fa-fb2b064f403b None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:59.899169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:59.912634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c0b69-630b-4444-9a77-c3f7f6d983eb None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:59.938926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:59.955291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad3251-f629-4c9c-9518-f3e036cff026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:59.955918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad3251-f629-4c9c-9518-f3e036cff026 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:59.956398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad3251-f629-4c9c-9518-f3e036cff026 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:59.956692 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad3251-f629-4c9c-9518-f3e036cff026 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:59.957340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad3251-f629-4c9c-9518-f3e036cff026 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:59.957641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad3251-f629-4c9c-9518-f3e036cff026 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:59.959019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad3251-f629-4c9c-9518-f3e036cff026 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:59.971141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 428/845] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:38:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:59.971141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341330e-66a4-4360-bbdd-50ea0e285e24 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:59.971141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341330e-66a4-4360-bbdd-50ea0e285e24 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:59.971141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341330e-66a4-4360-bbdd-50ea0e285e24 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:59.971141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 429/846] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:38:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:59.972353 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a35ed2-a708-4b93-9bfc-ea788bf44c8d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:59.972629 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a35ed2-a708-4b93-9bfc-ea788bf44c8d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:59.972902 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a35ed2-a708-4b93-9bfc-ea788bf44c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:59.997477 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-70a35ed2-a708-4b93-9bfc-ea788bf44c8d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:59.998822 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-70a35ed2-a708-4b93-9bfc-ea788bf44c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:59.999460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c00152-7691-4927-a9fa-fb2b064f403b None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:00.039423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a60d5-59ec-4a47-9486-45e5c0a6d5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:00.040140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a60d5-59ec-4a47-9486-45e5c0a6d5df None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:00.040423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a60d5-59ec-4a47-9486-45e5c0a6d5df None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:00.040694 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a60d5-59ec-4a47-9486-45e5c0a6d5df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:00.091254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 430/847] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:38:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:39:00.129820 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a60d5-59ec-4a47-9486-45e5c0a6d5df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:00.131233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a60d5-59ec-4a47-9486-45e5c0a6d5df None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:00.157576 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:00.250559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 418/848] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:38:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 14:39:00.255890 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a35ed2-a708-4b93-9bfc-ea788bf44c8d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:00.369403 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a35ed2-a708-4b93-9bfc-ea788bf44c8d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:00.375642 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a60d5-59ec-4a47-9486-45e5c0a6d5df None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:00.386647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:00.411730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba7659-f165-4dec-83c7-ba4f818f7b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:00.412310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba7659-f165-4dec-83c7-ba4f818f7b19 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:00.412590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba7659-f165-4dec-83c7-ba4f818f7b19 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:00.412862 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba7659-f165-4dec-83c7-ba4f818f7b19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:00.424583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba7659-f165-4dec-83c7-ba4f818f7b19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:00.426050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba7659-f165-4dec-83c7-ba4f818f7b19 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:00.442412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-76ba7659-f165-4dec-83c7-ba4f818f7b19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:00.442412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:00.442412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:00.442412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:00.442412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:00.442412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:00.442412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:00.442412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:00.442412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:00.442412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:00.442412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:00.442412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:00.442412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:39:00.442412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:39:00.442412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:39:00.442412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:00.443455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:39:00.444352 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:39:00.444352 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:00.444352 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:00.444352 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 431/849] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:00.452105 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba7659-f165-4dec-83c7-ba4f818f7b19 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:00.502862 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a60d5-59ec-4a47-9486-45e5c0a6d5df None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:00.525881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:00.537448 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b189d-0272-4334-9e0b-a53d82764db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:00.538949 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b189d-0272-4334-9e0b-a53d82764db2 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:00.538949 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b189d-0272-4334-9e0b-a53d82764db2 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:00.538949 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b189d-0272-4334-9e0b-a53d82764db2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:00.549286 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b189d-0272-4334-9e0b-a53d82764db2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:39:00.554196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-aa1b189d-0272-4334-9e0b-a53d82764db2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:39:00.554196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:00.554196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:00.554196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:00.554196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:00.554196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:00.554196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:00.554196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:00.554196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:00.554196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:00.554196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:00.554196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:00.554196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:39:00.554196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:39:00.554196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:39:00.554196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:39:00.555342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:39:00.555342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:39:00.555342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:39:00.555342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:39:00.555342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:00.555342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:00.555342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:00.555342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:00.555342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:00.555342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:00.555342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:00.555342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:00.555342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:00.555342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:00.555342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:00.555342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:00.556385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:00.556385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:00.556385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:00.556385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:00.556385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:39:00.556385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:39:00.556385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:39:00.556385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:39:00.556385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:39:00.556385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:39:00.556385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:39:00.556385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:00.558866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 419/850] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:39:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:00.560449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20720ed5-99ba-488f-ad2b-f9c8ba0c7383 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:00.560714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20720ed5-99ba-488f-ad2b-f9c8ba0c7383 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:00.560977 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20720ed5-99ba-488f-ad2b-f9c8ba0c7383 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:00.562358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 420/851] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:00.566997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20720ed5-99ba-488f-ad2b-f9c8ba0c7383 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:00.570943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba7659-f165-4dec-83c7-ba4f818f7b19 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:00.607745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00754f-0847-4b52-a1b9-e6b3727ac692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:00.608360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00754f-0847-4b52-a1b9-e6b3727ac692 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:00.608688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00754f-0847-4b52-a1b9-e6b3727ac692 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:00.608911 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00754f-0847-4b52-a1b9-e6b3727ac692 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:00.609686 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00754f-0847-4b52-a1b9-e6b3727ac692 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:00.609996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00754f-0847-4b52-a1b9-e6b3727ac692 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:00.611556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00754f-0847-4b52-a1b9-e6b3727ac692 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:00.628487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 432/852] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:00.628487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9732ad-76ba-4c7a-ab8e-e92b337990eb None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:00.628487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9732ad-76ba-4c7a-ab8e-e92b337990eb None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:00.628487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9732ad-76ba-4c7a-ab8e-e92b337990eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:00.646200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-9d9732ad-76ba-4c7a-ab8e-e92b337990eb None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:00.647360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9d9732ad-76ba-4c7a-ab8e-e92b337990eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:00.669699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:00.675040 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20720ed5-99ba-488f-ad2b-f9c8ba0c7383 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:00.708948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8fe60-4b11-4ea9-8c6f-13f16f33d9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:00.709548 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8fe60-4b11-4ea9-8c6f-13f16f33d9b8 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:00.709828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8fe60-4b11-4ea9-8c6f-13f16f33d9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:00.710122 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8fe60-4b11-4ea9-8c6f-13f16f33d9b8 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:00.710775 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8fe60-4b11-4ea9-8c6f-13f16f33d9b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:00.711080 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8fe60-4b11-4ea9-8c6f-13f16f33d9b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:00.712458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8fe60-4b11-4ea9-8c6f-13f16f33d9b8 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:00.720285 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 421/853] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:39:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:00.720927 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 433/854] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:00.723610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8fe60-4b11-4ea9-8c6f-13f16f33d9b8 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:00.725721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9732ad-76ba-4c7a-ab8e-e92b337990eb None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:00.781608 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:00.835453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9732ad-76ba-4c7a-ab8e-e92b337990eb None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:00.848365 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8fe60-4b11-4ea9-8c6f-13f16f33d9b8 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:00.875885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f8597-8638-4e5f-ab2c-97dbe619ba59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:00.876929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f8597-8638-4e5f-ab2c-97dbe619ba59 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:00.877432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f8597-8638-4e5f-ab2c-97dbe619ba59 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:00.877979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f8597-8638-4e5f-ab2c-97dbe619ba59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:00.889647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f8597-8638-4e5f-ab2c-97dbe619ba59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:00.891810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5f87c-a532-44da-b8c7-03c8d059107c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:00.892189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f8597-8638-4e5f-ab2c-97dbe619ba59 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:00.892430 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5f87c-a532-44da-b8c7-03c8d059107c None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:00.892703 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5f87c-a532-44da-b8c7-03c8d059107c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:00.892966 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5f87c-a532-44da-b8c7-03c8d059107c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:00.901217 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5f87c-a532-44da-b8c7-03c8d059107c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:00.902937 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5f87c-a532-44da-b8c7-03c8d059107c None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:00.910365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-0e5f8597-8638-4e5f-ab2c-97dbe619ba59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:00.910365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:00.910365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:00.910365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:00.910365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:00.910365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:00.910365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:00.910365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:00.910365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:00.910365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:00.910365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:00.910365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:00.910365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:39:00.910365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:39:00.910365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:39:00.910365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:00.911372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:39:00.912306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:39:00.912306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:00.912306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:00.914453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 434/855] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:00.920124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-12c5f87c-a532-44da-b8c7-03c8d059107c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:00.920124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:00.920124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:00.920124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:00.920124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:00.920124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:00.920124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:00.920124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:00.920124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:00.920124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:00.920124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:00.920124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:00.920124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:39:00.920124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:39:00.920124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:00.921042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:00.921963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:39:00.921963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:39:00.921963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:39:00.921963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:39:00.921963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:39:00.921963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:39:00.921963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:00.921963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:00.921963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 422/856] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:39:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:00.923035 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f8597-8638-4e5f-ab2c-97dbe619ba59 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:00.927037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5f87c-a532-44da-b8c7-03c8d059107c None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:00.937133 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:01.042616 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5f87c-a532-44da-b8c7-03c8d059107c None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:01.045982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:01.054614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f8597-8638-4e5f-ab2c-97dbe619ba59 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:01.084709 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb528e-c400-498c-bbf8-e7bb9ca5512a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:01.085259 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb528e-c400-498c-bbf8-e7bb9ca5512a None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:01.085540 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb528e-c400-498c-bbf8-e7bb9ca5512a None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:01.086060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb528e-c400-498c-bbf8-e7bb9ca5512a None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:01.086492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb528e-c400-498c-bbf8-e7bb9ca5512a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:01.086790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb528e-c400-498c-bbf8-e7bb9ca5512a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:01.088786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb528e-c400-498c-bbf8-e7bb9ca5512a None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:01.091570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24733c09-103a-429d-8d34-e00f340c8321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:01.092172 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24733c09-103a-429d-8d34-e00f340c8321 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:01.092451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24733c09-103a-429d-8d34-e00f340c8321 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:01.092723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24733c09-103a-429d-8d34-e00f340c8321 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:01.093371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24733c09-103a-429d-8d34-e00f340c8321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:01.093675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24733c09-103a-429d-8d34-e00f340c8321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:01.095034 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24733c09-103a-429d-8d34-e00f340c8321 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:01.106228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 423/857] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:39:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:01.106228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb528e-c400-498c-bbf8-e7bb9ca5512a None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:01.112016 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 435/858] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:01.116518 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24733c09-103a-429d-8d34-e00f340c8321 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:01.182911 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:01.213313 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb528e-c400-498c-bbf8-e7bb9ca5512a None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:01.217077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24733c09-103a-429d-8d34-e00f340c8321 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:01.248908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101aafb-a36c-450a-9099-e6f0026ef184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:01.249521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101aafb-a36c-450a-9099-e6f0026ef184 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:01.249801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101aafb-a36c-450a-9099-e6f0026ef184 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:01.250069 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101aafb-a36c-450a-9099-e6f0026ef184 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:01.250649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101aafb-a36c-450a-9099-e6f0026ef184 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:01.252034 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101aafb-a36c-450a-9099-e6f0026ef184 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:01.256257 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8f8e1-5309-468b-9e38-9aaebe9e1a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:01.256848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8f8e1-5309-468b-9e38-9aaebe9e1a8c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:01.257123 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8f8e1-5309-468b-9e38-9aaebe9e1a8c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:01.257391 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8f8e1-5309-468b-9e38-9aaebe9e1a8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:01.258111 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8f8e1-5309-468b-9e38-9aaebe9e1a8c None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=bc23480d5aa94cad9e6599762ce01548) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:01.258416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8f8e1-5309-468b-9e38-9aaebe9e1a8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:01.260438 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8f8e1-5309-468b-9e38-9aaebe9e1a8c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:01.261288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-0101aafb-a36c-450a-9099-e6f0026ef184 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:39:01.261288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:01.261288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:39:01.261288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:39:01.261288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:39:01.261288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:39:01.261288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:39:01.261288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:01.261288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:39:01.261288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:01.261288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:01.261288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:01.261288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:01.261288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:01.261288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:01.261288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:01.262183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:01.263093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:01.263093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:01.263093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:01.263093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:01.263093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:01.263093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:01.263093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:01.263093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:01.263093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:01.263093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:01.263093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:39:01.263093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:01.263093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:39:01.263093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:01.267534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 424/859] 10.222.0.27 () {56 vars in 1132 bytes} [Tue Aug 30 14:39:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:01.268151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101aafb-a36c-450a-9099-e6f0026ef184 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:01.279680 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 436/860] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:39:01 2022] GET /identity/v3/role_assignments?user.id=bc23480d5aa94cad9e6599762ce01548&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:01.279680 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8f8e1-5309-468b-9e38-9aaebe9e1a8c None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:01.315211 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:01.404320 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8f8e1-5309-468b-9e38-9aaebe9e1a8c None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:01.409662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101aafb-a36c-450a-9099-e6f0026ef184 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:01.438719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:01.445063 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849bd72-87a7-4a36-a321-558c5d8b562c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:01.445999 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849bd72-87a7-4a36-a321-558c5d8b562c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:01.446388 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849bd72-87a7-4a36-a321-558c5d8b562c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:01.446674 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849bd72-87a7-4a36-a321-558c5d8b562c None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:01.447455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849bd72-87a7-4a36-a321-558c5d8b562c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:01.447852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849bd72-87a7-4a36-a321-558c5d8b562c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:01.455729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849bd72-87a7-4a36-a321-558c5d8b562c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:01.455847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7d04a-bd18-490c-bf08-c6b165e1b0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:01.455847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7d04a-bd18-490c-bf08-c6b165e1b0f8 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:01.455847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7d04a-bd18-490c-bf08-c6b165e1b0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:01.455847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7d04a-bd18-490c-bf08-c6b165e1b0f8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:01.456223 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 437/861] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 30 14:39:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:01.459969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7d04a-bd18-490c-bf08-c6b165e1b0f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:01.460343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849bd72-87a7-4a36-a321-558c5d8b562c None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:01.461524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7d04a-bd18-490c-bf08-c6b165e1b0f8 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:01.476387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-59b7d04a-bd18-490c-bf08-c6b165e1b0f8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:39:01.476387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:01.476387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:01.476387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:01.476387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:01.476387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:01.476387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:01.476387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:01.476387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:01.476387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:01.476387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:01.476387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:01.476387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:39:01.476387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:39:01.476387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:39:01.476387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:01.477552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:01.479235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:01.479235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:01.479235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:39:01.479235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:39:01.479235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:39:01.479235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:39:01.479235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:39:01.479235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:39:01.479235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:39:01.479235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:01.482747 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 425/862] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 30 14:39:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:01.484047 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7d04a-bd18-490c-bf08-c6b165e1b0f8 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:01.579478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:01.592174 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7d04a-bd18-490c-bf08-c6b165e1b0f8 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:01.609145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849bd72-87a7-4a36-a321-558c5d8b562c None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:01.627563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ded85-655a-4a70-bf51-e46717e893cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:01.628265 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ded85-655a-4a70-bf51-e46717e893cf None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:01.628607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ded85-655a-4a70-bf51-e46717e893cf None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:01.628992 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ded85-655a-4a70-bf51-e46717e893cf None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:01.629825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ded85-655a-4a70-bf51-e46717e893cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:01.630342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ded85-655a-4a70-bf51-e46717e893cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:01.632060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ded85-655a-4a70-bf51-e46717e893cf None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:01.651286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025b682-521b-4423-9413-d8f5744c49ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:01.651898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025b682-521b-4423-9413-d8f5744c49ec None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:01.652169 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025b682-521b-4423-9413-d8f5744c49ec None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:01.652436 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025b682-521b-4423-9413-d8f5744c49ec None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:01.653071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025b682-521b-4423-9413-d8f5744c49ec None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:01.653365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025b682-521b-4423-9413-d8f5744c49ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:01.654730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025b682-521b-4423-9413-d8f5744c49ec None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:01.659864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 438/864] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 30 14:39:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:01.660034 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 426/864] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Aug 30 14:39:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:01.661827 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025b682-521b-4423-9413-d8f5744c49ec None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:01.735090 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:01.782428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025b682-521b-4423-9413-d8f5744c49ec None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:01.820654 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397c839-628c-4e3b-b297-884266117b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:01.821455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397c839-628c-4e3b-b297-884266117b15 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:01.821833 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397c839-628c-4e3b-b297-884266117b15 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:01.822813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397c839-628c-4e3b-b297-884266117b15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:01.893892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397c839-628c-4e3b-b297-884266117b15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:01.895620 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397c839-628c-4e3b-b297-884266117b15 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:01.941423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d195cf-f757-41d3-ab96-e8807ee0b4d1 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:01.941942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d195cf-f757-41d3-ab96-e8807ee0b4d1 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:01.941942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d195cf-f757-41d3-ab96-e8807ee0b4d1 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:01.943622 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 427/865] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:39:01.949425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc2e62-f329-4a43-900e-4bb4113e9363 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:01.949715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc2e62-f329-4a43-900e-4bb4113e9363 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:01.949993 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc2e62-f329-4a43-900e-4bb4113e9363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:01.958078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:01.971334 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-89fc2e62-f329-4a43-900e-4bb4113e9363 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:01.973923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-89fc2e62-f329-4a43-900e-4bb4113e9363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:02.006991 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 439/866] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:39:01 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 14:39:02.012822 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397c839-628c-4e3b-b297-884266117b15 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:02.075222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 428/867] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:39:02.127056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397c839-628c-4e3b-b297-884266117b15 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:02.162932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e4729-553c-4182-ba6c-18e1bf4dfac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:02.163610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e4729-553c-4182-ba6c-18e1bf4dfac7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:02.163890 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e4729-553c-4182-ba6c-18e1bf4dfac7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:02.164163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e4729-553c-4182-ba6c-18e1bf4dfac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:02.174919 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e4729-553c-4182-ba6c-18e1bf4dfac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:02.176558 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e4729-553c-4182-ba6c-18e1bf4dfac7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:02.195504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-3a4e4729-553c-4182-ba6c-18e1bf4dfac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:02.195504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:02.195504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:02.195504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:02.195504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:02.195504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:02.195504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:02.195504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:02.195504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:02.195504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:02.195504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:02.195504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:02.195504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:39:02.195504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:39:02.195504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:39:02.195504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:02.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:39:02.197430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:39:02.197430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:02.197430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:02.199505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 440/868] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:02.205342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc2e62-f329-4a43-900e-4bb4113e9363 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:02.284661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:02.308926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc2e62-f329-4a43-900e-4bb4113e9363 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:02.352160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7b04e-7cd3-4754-94dd-cff2a657b97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:02.352857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7b04e-7cd3-4754-94dd-cff2a657b97b None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:02.353143 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7b04e-7cd3-4754-94dd-cff2a657b97b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:02.353930 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7b04e-7cd3-4754-94dd-cff2a657b97b None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:02.354580 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7b04e-7cd3-4754-94dd-cff2a657b97b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:02.354879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7b04e-7cd3-4754-94dd-cff2a657b97b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:02.356116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e4729-553c-4182-ba6c-18e1bf4dfac7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:02.356285 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7b04e-7cd3-4754-94dd-cff2a657b97b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:02.368866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 429/869] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:02.376287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7b04e-7cd3-4754-94dd-cff2a657b97b None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:02.378083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:02.468126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e4729-553c-4182-ba6c-18e1bf4dfac7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:02.489623 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7b04e-7cd3-4754-94dd-cff2a657b97b None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:02.505732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1e614-55e3-4ef2-a098-898436dd7fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:02.506336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1e614-55e3-4ef2-a098-898436dd7fae None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:02.506612 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1e614-55e3-4ef2-a098-898436dd7fae None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:02.506877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1e614-55e3-4ef2-a098-898436dd7fae None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:02.507556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1e614-55e3-4ef2-a098-898436dd7fae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:02.508913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1e614-55e3-4ef2-a098-898436dd7fae None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:02.531205 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ef93b-3c91-454a-8044-4941bb4ed77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:02.531205 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ef93b-3c91-454a-8044-4941bb4ed77d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:02.531205 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ef93b-3c91-454a-8044-4941bb4ed77d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:02.531205 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ef93b-3c91-454a-8044-4941bb4ed77d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:02.531555 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:02.536027 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ef93b-3c91-454a-8044-4941bb4ed77d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:02.537650 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ef93b-3c91-454a-8044-4941bb4ed77d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:02.549774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-db3ef93b-3c91-454a-8044-4941bb4ed77d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:02.549774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:02.549774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:02.549774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:02.549774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:02.549774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:02.549774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:02.549774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:02.549774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:02.549774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:02.549774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:02.549774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:02.549774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:39:02.549774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:39:02.549774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:02.550741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:02.551658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:39:02.551658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:39:02.551658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:39:02.551658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:39:02.551658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:39:02.551658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:39:02.551658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:02.551658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:02.555180 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 430/870] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:39:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:02.557271 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ef93b-3c91-454a-8044-4941bb4ed77d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:02.592084 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 441/871] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 30 14:39:02 2022] POST /identity/v3/services => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:39:02.674554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ef93b-3c91-454a-8044-4941bb4ed77d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:02.720470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b432b-bef6-4d75-be52-13885152fbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:02.721424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b432b-bef6-4d75-be52-13885152fbd2 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:02.721884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b432b-bef6-4d75-be52-13885152fbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:02.722333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b432b-bef6-4d75-be52-13885152fbd2 None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:02.723303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b432b-bef6-4d75-be52-13885152fbd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:02.723810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b432b-bef6-4d75-be52-13885152fbd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:02.725501 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b432b-bef6-4d75-be52-13885152fbd2 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:02.739988 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 431/872] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:39:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:02.745697 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1e614-55e3-4ef2-a098-898436dd7fae None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:02.824889 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:02.857572 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1e614-55e3-4ef2-a098-898436dd7fae None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:02.893185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac611176-9cce-4c45-83f9-f9799af4c98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:02.894325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac611176-9cce-4c45-83f9-f9799af4c98c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:02.894958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac611176-9cce-4c45-83f9-f9799af4c98c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:02.895259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac611176-9cce-4c45-83f9-f9799af4c98c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:02.895995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac611176-9cce-4c45-83f9-f9799af4c98c None admin] RBAC: Adding query filter params (role.id=244ad93ecd1949c9a6e8f74b135b43b4, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=912e9c732f194fa496a9111bbfab3efa) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:02.896539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac611176-9cce-4c45-83f9-f9799af4c98c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:02.898568 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac611176-9cce-4c45-83f9-f9799af4c98c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:02.905976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 442/873] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:39:02 2022] GET /identity/v3/role_assignments?user.id=912e9c732f194fa496a9111bbfab3efa&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=244ad93ecd1949c9a6e8f74b135b43b4 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:03.288916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef86e7-e850-4119-ae31-b2d4ebd250db None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:03.289440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef86e7-e850-4119-ae31-b2d4ebd250db None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:03.289515 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef86e7-e850-4119-ae31-b2d4ebd250db None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:03.291141 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 432/874] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:03.299333 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714c507-f25f-4c04-bed9-73d1ccaaf51d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:03.299842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714c507-f25f-4c04-bed9-73d1ccaaf51d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:03.299842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714c507-f25f-4c04-bed9-73d1ccaaf51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:03.318997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e714c507-f25f-4c04-bed9-73d1ccaaf51d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:03.320239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e714c507-f25f-4c04-bed9-73d1ccaaf51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:03.390561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 443/875] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:03 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:03.651256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef86e7-e850-4119-ae31-b2d4ebd250db None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:03.694865 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100bd2d-7f10-45b1-8bee-5b754bf5eeeb None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:03.695210 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100bd2d-7f10-45b1-8bee-5b754bf5eeeb None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:03.695463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100bd2d-7f10-45b1-8bee-5b754bf5eeeb None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:03.697005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 444/876] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:39:03.703126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ddae7-3135-4af1-990c-7bf4751d978a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:03.703423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ddae7-3135-4af1-990c-7bf4751d978a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:03.703699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ddae7-3135-4af1-990c-7bf4751d978a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:03.716364 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:03.727482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2e2ddae7-3135-4af1-990c-7bf4751d978a None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:03.729221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2e2ddae7-3135-4af1-990c-7bf4751d978a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:03.780515 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef86e7-e850-4119-ae31-b2d4ebd250db None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:03.795624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:03.802457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 445/877] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:03 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:39:03.825400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6bed0-28a2-47b1-bbd1-d98c6f6d6637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:03.826376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6bed0-28a2-47b1-bbd1-d98c6f6d6637 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:03.826850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6bed0-28a2-47b1-bbd1-d98c6f6d6637 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:03.827321 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6bed0-28a2-47b1-bbd1-d98c6f6d6637 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:03.828079 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6bed0-28a2-47b1-bbd1-d98c6f6d6637 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:03.829604 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6bed0-28a2-47b1-bbd1-d98c6f6d6637 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-8fc6bed0-28a2-47b1-bbd1-d98c6f6d6637 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:03.838850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:03.839855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:03.839855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:03.839855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:03.839855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:03.839855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:03.839855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:03.839855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:39:03.839855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:39:03.839855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:39:03.839855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:39:03.839855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:03.839855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:03.839855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:03.839855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:03.839855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:03.839855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:03.840699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:03.840699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:03.840699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:03.840699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:03.840699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:03.840699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:03.840699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:03.840699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:03.840699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:03.840699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:03.840699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:39:03.840699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:03.840699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:39:03.840699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:03.843050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 433/878] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 30 14:39:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:03.848523 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ddae7-3135-4af1-990c-7bf4751d978a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:03.951966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ddae7-3135-4af1-990c-7bf4751d978a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:03.992006 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091864c-9118-4c98-b208-d628bc2bd0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:03.992608 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091864c-9118-4c98-b208-d628bc2bd0c8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:03.992884 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091864c-9118-4c98-b208-d628bc2bd0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:03.993149 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091864c-9118-4c98-b208-d628bc2bd0c8 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:03.993801 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091864c-9118-4c98-b208-d628bc2bd0c8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:03.994099 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091864c-9118-4c98-b208-d628bc2bd0c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:03.995495 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091864c-9118-4c98-b208-d628bc2bd0c8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:04.000398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 446/879] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 30 14:39:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:04.006244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6bed0-28a2-47b1-bbd1-d98c6f6d6637 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:04.114321 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6bed0-28a2-47b1-bbd1-d98c6f6d6637 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:04.128524 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091864c-9118-4c98-b208-d628bc2bd0c8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:04.135395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:04.155837 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d76101-904e-48f3-9bbc-a62cfad02ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:04.156391 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d76101-904e-48f3-9bbc-a62cfad02ff6 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:04.156662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d76101-904e-48f3-9bbc-a62cfad02ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:04.156932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d76101-904e-48f3-9bbc-a62cfad02ff6 None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:04.157563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d76101-904e-48f3-9bbc-a62cfad02ff6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:04.157863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d76101-904e-48f3-9bbc-a62cfad02ff6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:04.159289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d76101-904e-48f3-9bbc-a62cfad02ff6 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:04.164298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 434/880] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 30 14:39:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:04.203648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:04.250434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091864c-9118-4c98-b208-d628bc2bd0c8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:04.289431 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32a0cf-a9e8-4073-892d-64e051af59a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:04.290106 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32a0cf-a9e8-4073-892d-64e051af59a4 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:04.290393 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32a0cf-a9e8-4073-892d-64e051af59a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:04.290704 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32a0cf-a9e8-4073-892d-64e051af59a4 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:04.291445 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32a0cf-a9e8-4073-892d-64e051af59a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:04.292830 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32a0cf-a9e8-4073-892d-64e051af59a4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:04.391663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 447/881] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 30 14:39:04 2022] POST /identity/v3/services => generated 269 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:39:04.654178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c9d4d-ac94-41e6-9c1a-37d1bc6569f3 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:04.654701 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c9d4d-ac94-41e6-9c1a-37d1bc6569f3 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:04.655006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c9d4d-ac94-41e6-9c1a-37d1bc6569f3 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:04.656744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 435/882] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:04.661867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74244a88-f49e-4083-a7db-907135e35cec None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:04.662259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74244a88-f49e-4083-a7db-907135e35cec None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:04.662632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74244a88-f49e-4083-a7db-907135e35cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:04.680846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-74244a88-f49e-4083-a7db-907135e35cec None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:04.682550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-74244a88-f49e-4083-a7db-907135e35cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:04.758115 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 448/883] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:04.942347 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b5076-806f-4251-9360-bdc11ea35d46 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:04.942974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b5076-806f-4251-9360-bdc11ea35d46 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:04.943481 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b5076-806f-4251-9360-bdc11ea35d46 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:04.945292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 436/884] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:04.951575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477d420-8944-4230-9980-abad72717b6d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:04.952095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477d420-8944-4230-9980-abad72717b6d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:04.952601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477d420-8944-4230-9980-abad72717b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:04.970826 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2477d420-8944-4230-9980-abad72717b6d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:04.972315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2477d420-8944-4230-9980-abad72717b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:05.014841 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:05.028878 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b5076-806f-4251-9360-bdc11ea35d46 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:05.045509 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 449/885] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:05.140056 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b5076-806f-4251-9360-bdc11ea35d46 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:05.176301 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd13db-c070-43c2-aa7f-9af8f48310f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:05.176933 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd13db-c070-43c2-aa7f-9af8f48310f4 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:05.177212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd13db-c070-43c2-aa7f-9af8f48310f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:05.177484 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd13db-c070-43c2-aa7f-9af8f48310f4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:05.178127 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd13db-c070-43c2-aa7f-9af8f48310f4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:05.179596 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd13db-c070-43c2-aa7f-9af8f48310f4 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-03fd13db-c070-43c2-aa7f-9af8f48310f4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:05.189425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:05.190419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:05.190419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:05.190419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:05.190419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:05.190419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:05.190419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:05.190419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:39:05.190419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:39:05.190419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:39:05.190419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:39:05.190419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:05.190419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:05.190419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:05.190419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:05.190419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:05.190419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:05.191319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:05.191319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:05.191319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:05.191319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:05.191319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:05.191319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:05.191319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:05.191319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:05.191319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:05.191319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:05.191319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:39:05.191319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:05.191319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:39:05.191319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:05.191319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 437/886] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:39:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:05.196779 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477d420-8944-4230-9980-abad72717b6d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:05.301770 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477d420-8944-4230-9980-abad72717b6d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:05.324259 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd13db-c070-43c2-aa7f-9af8f48310f4 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:05.343572 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e1119-04c0-455a-b13e-a39dad0457c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:05.344577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e1119-04c0-455a-b13e-a39dad0457c5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:05.345063 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e1119-04c0-455a-b13e-a39dad0457c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:05.345544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e1119-04c0-455a-b13e-a39dad0457c5 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:05.346403 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e1119-04c0-455a-b13e-a39dad0457c5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:05.346926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e1119-04c0-455a-b13e-a39dad0457c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:05.348532 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e1119-04c0-455a-b13e-a39dad0457c5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:05.354776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 450/887] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:39:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:05.360602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e1119-04c0-455a-b13e-a39dad0457c5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:05.385247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:05.414950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:05.455045 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd13db-c070-43c2-aa7f-9af8f48310f4 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:05.475133 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e1119-04c0-455a-b13e-a39dad0457c5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:05.494906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7128e-8f6c-4cff-9fbb-d6690650c4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:05.495968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7128e-8f6c-4cff-9fbb-d6690650c4ec None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:05.496583 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7128e-8f6c-4cff-9fbb-d6690650c4ec None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:05.497060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7128e-8f6c-4cff-9fbb-d6690650c4ec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:05.509861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7128e-8f6c-4cff-9fbb-d6690650c4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:39:05.512348 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1c45c-a55f-4a2f-bd3e-384ee16441d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:05.512963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1c45c-a55f-4a2f-bd3e-384ee16441d0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:05.513244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1c45c-a55f-4a2f-bd3e-384ee16441d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:05.513518 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1c45c-a55f-4a2f-bd3e-384ee16441d0 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:05.514168 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1c45c-a55f-4a2f-bd3e-384ee16441d0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:05.514476 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1c45c-a55f-4a2f-bd3e-384ee16441d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:05.515878 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1c45c-a55f-4a2f-bd3e-384ee16441d0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:05.517661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-8ee7128e-8f6c-4cff-9fbb-d6690650c4ec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:39:05.517661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:05.517661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:05.517661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:05.517661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:05.517661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:05.517661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:05.517661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:05.517661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:05.517661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:05.517661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:05.517661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:05.517661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:39:05.517661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:39:05.517661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:39:05.517661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:39:05.518562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:39:05.518562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:39:05.518562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:39:05.518562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:39:05.518562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:05.518562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:05.518562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:05.518562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:05.518562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:05.518562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:05.518562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:05.518562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:05.518562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:05.518562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:05.518562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:05.518562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:05.519462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:05.519462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:05.519462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:05.519462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:05.519462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:39:05.519462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:39:05.519462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:39:05.519462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:39:05.519462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:39:05.519462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:39:05.519462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:39:05.519462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:05.526526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 451/888] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:39:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:05.527173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 438/889] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Aug 30 14:39:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:05.530580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1c45c-a55f-4a2f-bd3e-384ee16441d0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:05.531787 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7128e-8f6c-4cff-9fbb-d6690650c4ec None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:05.634811 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1c45c-a55f-4a2f-bd3e-384ee16441d0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:05.638084 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:05.643451 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7128e-8f6c-4cff-9fbb-d6690650c4ec None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:05.645747 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:05.672305 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733efda-f33c-4143-b67b-7267883c93dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:05.673351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733efda-f33c-4143-b67b-7267883c93dd None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:05.673840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733efda-f33c-4143-b67b-7267883c93dd None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:05.674317 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733efda-f33c-4143-b67b-7267883c93dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:05.675644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733efda-f33c-4143-b67b-7267883c93dd None admin] RBAC: Adding query filter params (interface=public, service_id=2e0166ac8d034fec98e1a3b4cafe8bea, region_id=RegionOne) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:05.676198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733efda-f33c-4143-b67b-7267883c93dd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:05.678066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733efda-f33c-4143-b67b-7267883c93dd None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:05.681447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd83672-c81e-4ba3-ad0b-ccb920c6a33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:05.681962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd83672-c81e-4ba3-ad0b-ccb920c6a33b None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:05.682213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd83672-c81e-4ba3-ad0b-ccb920c6a33b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:05.688740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd83672-c81e-4ba3-ad0b-ccb920c6a33b None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:05.688740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd83672-c81e-4ba3-ad0b-ccb920c6a33b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:05.688740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd83672-c81e-4ba3-ad0b-ccb920c6a33b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:05.688740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd83672-c81e-4ba3-ad0b-ccb920c6a33b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:05.689042 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 452/890] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Aug 30 14:39:05 2022] GET /identity/v3/endpoints?service_id=2e0166ac8d034fec98e1a3b4cafe8bea&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:05.692508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733efda-f33c-4143-b67b-7267883c93dd None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:05.704382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 439/891] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Aug 30 14:39:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:05.705904 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd83672-c81e-4ba3-ad0b-ccb920c6a33b None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:05.825485 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd83672-c81e-4ba3-ad0b-ccb920c6a33b None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:05.830430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733efda-f33c-4143-b67b-7267883c93dd None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:05.863620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1070db-7212-47cb-be87-f2e3c9395740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:05.864653 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1070db-7212-47cb-be87-f2e3c9395740 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:05.865156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1070db-7212-47cb-be87-f2e3c9395740 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:05.865633 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1070db-7212-47cb-be87-f2e3c9395740 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:05.875302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53fded-ce8d-4f8d-b44d-ed4bfb3dd062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:05.875912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53fded-ce8d-4f8d-b44d-ed4bfb3dd062 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:05.876195 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53fded-ce8d-4f8d-b44d-ed4bfb3dd062 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:05.876466 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53fded-ce8d-4f8d-b44d-ed4bfb3dd062 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:05.877108 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53fded-ce8d-4f8d-b44d-ed4bfb3dd062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:05.880529 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53fded-ce8d-4f8d-b44d-ed4bfb3dd062 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:05.881317 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1070db-7212-47cb-be87-f2e3c9395740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:05.882969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1070db-7212-47cb-be87-f2e3c9395740 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:05.887183 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 453/892] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 30 14:39:05 2022] GET /identity/v3/services => generated 1268 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:39:05.905466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-7a1070db-7212-47cb-be87-f2e3c9395740 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:05.905466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:05.905466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:05.905466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:05.905466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:05.905466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:05.905466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:05.905466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:05.905466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:05.905466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:05.905466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:05.905466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:05.905466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:39:05.905466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:39:05.905466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:39:05.905466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:05.906491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:39:05.907854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:39:05.907854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:05.907854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:05.909845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 440/893] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:05.916742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53fded-ce8d-4f8d-b44d-ed4bfb3dd062 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:05.953367 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:06.030413 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53fded-ce8d-4f8d-b44d-ed4bfb3dd062 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:06.070162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e197a-3c0e-43f6-a398-6bcd1936eb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:06.070766 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e197a-3c0e-43f6-a398-6bcd1936eb4a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:06.071053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e197a-3c0e-43f6-a398-6bcd1936eb4a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:06.071363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e197a-3c0e-43f6-a398-6bcd1936eb4a None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:06.072050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e197a-3c0e-43f6-a398-6bcd1936eb4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:06.072354 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e197a-3c0e-43f6-a398-6bcd1936eb4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:06.073738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e197a-3c0e-43f6-a398-6bcd1936eb4a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:06.085734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 454/894] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:06.091914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1070db-7212-47cb-be87-f2e3c9395740 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:06.112804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:06.144132 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab641f8-5fcb-494f-b258-88ae4f635f0f None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:06.144132 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab641f8-5fcb-494f-b258-88ae4f635f0f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:06.144422 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab641f8-5fcb-494f-b258-88ae4f635f0f None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:06.146002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 455/895] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:39:06.151651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953c8a0-3438-4c39-b88d-dedcc8f00827 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:06.151929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953c8a0-3438-4c39-b88d-dedcc8f00827 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:06.152198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953c8a0-3438-4c39-b88d-dedcc8f00827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:06.171023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e953c8a0-3438-4c39-b88d-dedcc8f00827 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:06.172368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e953c8a0-3438-4c39-b88d-dedcc8f00827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:06.213940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1070db-7212-47cb-be87-f2e3c9395740 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:06.258045 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938df92d-6f55-4827-926e-56d7ef788216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:06.258419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 456/896] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:39:06.259132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938df92d-6f55-4827-926e-56d7ef788216 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:06.259598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938df92d-6f55-4827-926e-56d7ef788216 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:06.260014 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938df92d-6f55-4827-926e-56d7ef788216 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:06.268109 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938df92d-6f55-4827-926e-56d7ef788216 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:06.269766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938df92d-6f55-4827-926e-56d7ef788216 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:06.289114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-938df92d-6f55-4827-926e-56d7ef788216 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:39:06.289114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:06.289114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:06.289114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:06.289114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:06.289114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:06.289114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:06.289114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:06.289114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:06.289114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:06.289114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:06.289114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:06.289114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:39:06.289114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:39:06.289114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:39:06.289114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:06.290226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:06.291982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:06.291982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:06.291982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:39:06.291982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:39:06.291982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:39:06.291982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:39:06.291982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:39:06.291982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:39:06.291982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:39:06.291982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:06.291982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 441/897] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 30 14:39:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:06.293205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953c8a0-3438-4c39-b88d-dedcc8f00827 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:06.318927 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:06.336734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353394d-3ec7-44ad-83bc-530657eaead5 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:06.338451 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353394d-3ec7-44ad-83bc-530657eaead5 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:06.340324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353394d-3ec7-44ad-83bc-530657eaead5 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:06.342530 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 442/898] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:39:06.348035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb44b0d-c8e7-4a70-bb9d-fbab17cd0845 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:06.348317 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb44b0d-c8e7-4a70-bb9d-fbab17cd0845 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:06.348593 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb44b0d-c8e7-4a70-bb9d-fbab17cd0845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:06.376191 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-6eb44b0d-c8e7-4a70-bb9d-fbab17cd0845 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:06.377752 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6eb44b0d-c8e7-4a70-bb9d-fbab17cd0845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:06.431351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953c8a0-3438-4c39-b88d-dedcc8f00827 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:06.479977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55484041-33f3-4657-9ca2-a1ca406fdb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:06.480590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55484041-33f3-4657-9ca2-a1ca406fdb29 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:06.480868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55484041-33f3-4657-9ca2-a1ca406fdb29 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:06.481134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55484041-33f3-4657-9ca2-a1ca406fdb29 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:06.481881 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55484041-33f3-4657-9ca2-a1ca406fdb29 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:06.482216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55484041-33f3-4657-9ca2-a1ca406fdb29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:06.483761 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55484041-33f3-4657-9ca2-a1ca406fdb29 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:06.484849 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 443/899] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:39:06.510455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 457/900] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Aug 30 14:39:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:06.524527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb44b0d-c8e7-4a70-bb9d-fbab17cd0845 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:06.538914 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55484041-33f3-4657-9ca2-a1ca406fdb29 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:06.563754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:06.663487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:06.694733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55484041-33f3-4657-9ca2-a1ca406fdb29 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:06.703275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb44b0d-c8e7-4a70-bb9d-fbab17cd0845 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:06.744685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb230d-3f08-4194-b6ab-b290cbdedd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:06.745285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb230d-3f08-4194-b6ab-b290cbdedd2e None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:06.745561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb230d-3f08-4194-b6ab-b290cbdedd2e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:06.745828 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb230d-3f08-4194-b6ab-b290cbdedd2e None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:06.746488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb230d-3f08-4194-b6ab-b290cbdedd2e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:06.747873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb230d-3f08-4194-b6ab-b290cbdedd2e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:06.764611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a56ec8-07c2-4548-ab88-b91b3ee59a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:06.765233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a56ec8-07c2-4548-ab88-b91b3ee59a4b None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:06.765537 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a56ec8-07c2-4548-ab88-b91b3ee59a4b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:06.765811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a56ec8-07c2-4548-ab88-b91b3ee59a4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:06.839184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 458/901] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 30 14:39:06 2022] POST /identity/v3/services => generated 241 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:39:06.840723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb230d-3f08-4194-b6ab-b290cbdedd2e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:06.848225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a56ec8-07c2-4548-ab88-b91b3ee59a4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:06.849698 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a56ec8-07c2-4548-ab88-b91b3ee59a4b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:06.871129 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:06.998795 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 444/902] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:39:06 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:39:07.003427 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a56ec8-07c2-4548-ab88-b91b3ee59a4b None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:07.033524 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb230d-3f08-4194-b6ab-b290cbdedd2e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:07.095989 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bfabb-3f0b-494f-9e3a-4718832dadd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:07.096977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bfabb-3f0b-494f-9e3a-4718832dadd1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:07.097480 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bfabb-3f0b-494f-9e3a-4718832dadd1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:07.098418 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bfabb-3f0b-494f-9e3a-4718832dadd1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:07.099278 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bfabb-3f0b-494f-9e3a-4718832dadd1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:07.101700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bfabb-3f0b-494f-9e3a-4718832dadd1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:07.106273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:07.117273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-775bfabb-3f0b-494f-9e3a-4718832dadd1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:39:07.117273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:07.117273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:39:07.117273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:39:07.117273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:39:07.117273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:39:07.117273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:39:07.117273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:07.117273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:39:07.117273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:07.117273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:07.117273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:07.117273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:07.117273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:07.117273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:07.117273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:07.119533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:07.120435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:07.120435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:07.120435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:07.120435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:07.120435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:07.120435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:07.120435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:07.120435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:07.120435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:07.120435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:07.120435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:39:07.120435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:07.120435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:39:07.120435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:07.120435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 459/903] 10.222.0.27 () {56 vars in 1132 bytes} [Tue Aug 30 14:39:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:07.128395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bfabb-3f0b-494f-9e3a-4718832dadd1 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:07.129786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a56ec8-07c2-4548-ab88-b91b3ee59a4b None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:07.141758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:07.169755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788c3fb-ce83-44ab-bfe8-0b549df47e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:07.170407 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788c3fb-ce83-44ab-bfe8-0b549df47e40 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:07.170685 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788c3fb-ce83-44ab-bfe8-0b549df47e40 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:07.170951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788c3fb-ce83-44ab-bfe8-0b549df47e40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:07.181798 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788c3fb-ce83-44ab-bfe8-0b549df47e40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:07.183353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788c3fb-ce83-44ab-bfe8-0b549df47e40 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:07.200068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-c788c3fb-ce83-44ab-bfe8-0b549df47e40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:07.200068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:07.200068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:07.200068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:07.200068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:07.200068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:07.200068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:07.200068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:07.200068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:07.200068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:07.200068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:07.200068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:07.200068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:39:07.200068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:39:07.200068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:39:07.200068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:07.201060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:39:07.202525 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:39:07.202525 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:07.202525 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:07.202525 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 445/904] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:07.206671 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788c3fb-ce83-44ab-bfe8-0b549df47e40 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:07.283126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bfabb-3f0b-494f-9e3a-4718832dadd1 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:07.314440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788c3fb-ce83-44ab-bfe8-0b549df47e40 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:07.330882 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d37ce-62e9-47e2-ab62-66c1626748ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:07.332119 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d37ce-62e9-47e2-ab62-66c1626748ea None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:07.332626 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d37ce-62e9-47e2-ab62-66c1626748ea None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:07.333458 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d37ce-62e9-47e2-ab62-66c1626748ea None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:07.334310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d37ce-62e9-47e2-ab62-66c1626748ea None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:07.334877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d37ce-62e9-47e2-ab62-66c1626748ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:07.337015 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d37ce-62e9-47e2-ab62-66c1626748ea None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:07.345461 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 460/905] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 30 14:39:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:07.351686 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d37ce-62e9-47e2-ab62-66c1626748ea None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:07.356565 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fe0e7-6a84-4054-b4c8-192e9e483238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:07.356859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fe0e7-6a84-4054-b4c8-192e9e483238 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:07.357139 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fe0e7-6a84-4054-b4c8-192e9e483238 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:07.357409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fe0e7-6a84-4054-b4c8-192e9e483238 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:07.358054 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fe0e7-6a84-4054-b4c8-192e9e483238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:07.358350 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fe0e7-6a84-4054-b4c8-192e9e483238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:07.359741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fe0e7-6a84-4054-b4c8-192e9e483238 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:07.379128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 446/906] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:07.380914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fe0e7-6a84-4054-b4c8-192e9e483238 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:07.412553 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:07.432339 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:07.473825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d37ce-62e9-47e2-ab62-66c1626748ea None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:07.512709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed386fae-f44b-436a-8de4-34329dcc4cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:07.513325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed386fae-f44b-436a-8de4-34329dcc4cca None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:07.513604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed386fae-f44b-436a-8de4-34329dcc4cca None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:07.513877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed386fae-f44b-436a-8de4-34329dcc4cca None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:07.514618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed386fae-f44b-436a-8de4-34329dcc4cca None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:07.514935 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed386fae-f44b-436a-8de4-34329dcc4cca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:07.516431 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed386fae-f44b-436a-8de4-34329dcc4cca None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:07.522345 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 461/907] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 30 14:39:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:07.526853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fe0e7-6a84-4054-b4c8-192e9e483238 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:07.531233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed386fae-f44b-436a-8de4-34329dcc4cca None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:07.574717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e5bce-f425-4dbd-a47d-e8245fb20865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:07.575721 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e5bce-f425-4dbd-a47d-e8245fb20865 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:07.576168 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e5bce-f425-4dbd-a47d-e8245fb20865 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:07.576593 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e5bce-f425-4dbd-a47d-e8245fb20865 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:07.585092 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e5bce-f425-4dbd-a47d-e8245fb20865 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:07.586760 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e5bce-f425-4dbd-a47d-e8245fb20865 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:07.601203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-056e5bce-f425-4dbd-a47d-e8245fb20865 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:07.601203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:07.601203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:07.601203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:07.601203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:07.601203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:07.601203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:07.601203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:07.601203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:07.601203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:07.601203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:07.601203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:07.601203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:39:07.601203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:39:07.601203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:07.602362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:07.603499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:39:07.603499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:39:07.603499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:39:07.603499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:39:07.603499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:39:07.603499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:39:07.603499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:07.603499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:07.610361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 447/908] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:39:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:07.612018 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e5bce-f425-4dbd-a47d-e8245fb20865 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:07.641726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed386fae-f44b-436a-8de4-34329dcc4cca None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:07.669241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:07.677201 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b3c8e-0024-4739-9c21-d9bcaab1eb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:07.681724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b3c8e-0024-4739-9c21-d9bcaab1eb1f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:07.681724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b3c8e-0024-4739-9c21-d9bcaab1eb1f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:07.681724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b3c8e-0024-4739-9c21-d9bcaab1eb1f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:07.681724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b3c8e-0024-4739-9c21-d9bcaab1eb1f None admin] RBAC: Adding query filter params (interface=public, service_id=03fd6504640c4c54b142e77ce13e8eb9, region_id=RegionOne) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:07.681724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b3c8e-0024-4739-9c21-d9bcaab1eb1f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:07.681724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b3c8e-0024-4739-9c21-d9bcaab1eb1f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:07.691350 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 462/909] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Aug 30 14:39:07 2022] GET /identity/v3/endpoints?service_id=03fd6504640c4c54b142e77ce13e8eb9&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:07.692248 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b3c8e-0024-4739-9c21-d9bcaab1eb1f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:07.710177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:07.732971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e5bce-f425-4dbd-a47d-e8245fb20865 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:07.776393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e69c4-dcec-45f3-9bd5-8e7e7e8d0035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:07.777336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e69c4-dcec-45f3-9bd5-8e7e7e8d0035 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:07.777875 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e69c4-dcec-45f3-9bd5-8e7e7e8d0035 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:07.778293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e69c4-dcec-45f3-9bd5-8e7e7e8d0035 None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:07.779180 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e69c4-dcec-45f3-9bd5-8e7e7e8d0035 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:07.779654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e69c4-dcec-45f3-9bd5-8e7e7e8d0035 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:07.781445 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e69c4-dcec-45f3-9bd5-8e7e7e8d0035 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:07.796165 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b3c8e-0024-4739-9c21-d9bcaab1eb1f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:07.802782 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 448/910] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:39:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:07.804491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e69c4-dcec-45f3-9bd5-8e7e7e8d0035 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:07.832555 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedff1d7-831d-44e9-bbaa-a1510ddd8a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:07.833197 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedff1d7-831d-44e9-bbaa-a1510ddd8a64 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:07.833427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedff1d7-831d-44e9-bbaa-a1510ddd8a64 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:07.833870 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedff1d7-831d-44e9-bbaa-a1510ddd8a64 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:07.834690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedff1d7-831d-44e9-bbaa-a1510ddd8a64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:07.836053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedff1d7-831d-44e9-bbaa-a1510ddd8a64 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:07.851346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 463/911] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 30 14:39:07 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:39:07.923053 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e69c4-dcec-45f3-9bd5-8e7e7e8d0035 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:07.927405 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35842ae4-1b03-4e05-90f1-29efa85bb56b None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:07.927683 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35842ae4-1b03-4e05-90f1-29efa85bb56b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:07.928005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35842ae4-1b03-4e05-90f1-29efa85bb56b None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:07.929607 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 464/912] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:07.937452 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac99ce-a521-4169-9ea2-82d4605d5200 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:07.937727 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac99ce-a521-4169-9ea2-82d4605d5200 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:07.938103 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac99ce-a521-4169-9ea2-82d4605d5200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:07.958244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-28ac99ce-a521-4169-9ea2-82d4605d5200 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:07.959850 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-28ac99ce-a521-4169-9ea2-82d4605d5200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:07.965358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d089391-8272-463e-9171-95f44f3b8384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:07.966213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d089391-8272-463e-9171-95f44f3b8384 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:07.966638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d089391-8272-463e-9171-95f44f3b8384 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:07.967082 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d089391-8272-463e-9171-95f44f3b8384 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:07.967996 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d089391-8272-463e-9171-95f44f3b8384 None admin] RBAC: Adding query filter params (role.id=bcecc24215984de9a00652caa5433ffe, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=912e9c732f194fa496a9111bbfab3efa) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:07.968450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d089391-8272-463e-9171-95f44f3b8384 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:07.970198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d089391-8272-463e-9171-95f44f3b8384 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:07.983262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 449/913] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:39:07 2022] GET /identity/v3/role_assignments?user.id=912e9c732f194fa496a9111bbfab3efa&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=bcecc24215984de9a00652caa5433ffe => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:08.022322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:08.045186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 465/914] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:39:08.314756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d089391-8272-463e-9171-95f44f3b8384 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:08.318762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:08.419528 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d089391-8272-463e-9171-95f44f3b8384 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:08.453966 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dac8bb-640f-4081-9ffb-23097306f372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:08.454601 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dac8bb-640f-4081-9ffb-23097306f372 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:08.454878 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dac8bb-640f-4081-9ffb-23097306f372 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:08.455176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dac8bb-640f-4081-9ffb-23097306f372 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:08.455763 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dac8bb-640f-4081-9ffb-23097306f372 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:08.457127 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dac8bb-640f-4081-9ffb-23097306f372 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-87dac8bb-640f-4081-9ffb-23097306f372 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:08.466686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:08.467754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:08.467754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:08.467754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:08.467754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:08.467754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:08.467754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:08.467754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:39:08.467754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:39:08.467754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:39:08.467754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:39:08.467754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:08.467754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:08.467754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:08.467754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:08.467754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:08.467754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:08.468594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:08.468594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:08.468594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:08.468594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:08.468594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:08.468594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:08.468594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:08.468594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:08.468594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:08.468594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:08.468594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:39:08.468594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:08.468594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:39:08.468594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:08.470416 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 450/915] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:39:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:08.475384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac99ce-a521-4169-9ea2-82d4605d5200 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:08.578815 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac99ce-a521-4169-9ea2-82d4605d5200 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:08.613019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca68264-b761-431f-96cc-21dbc717f8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:08.613627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca68264-b761-431f-96cc-21dbc717f8c5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:08.613903 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca68264-b761-431f-96cc-21dbc717f8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:08.614171 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca68264-b761-431f-96cc-21dbc717f8c5 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:08.614797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca68264-b761-431f-96cc-21dbc717f8c5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:08.615112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca68264-b761-431f-96cc-21dbc717f8c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:08.616488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca68264-b761-431f-96cc-21dbc717f8c5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:08.622077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 466/916] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:39:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:08.627432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dac8bb-640f-4081-9ffb-23097306f372 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:08.696260 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:08.730396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dac8bb-640f-4081-9ffb-23097306f372 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:08.771010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74250f34-9cea-457f-9cf3-75b9a212a04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:08.772040 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74250f34-9cea-457f-9cf3-75b9a212a04b None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:08.772521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74250f34-9cea-457f-9cf3-75b9a212a04b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:08.772989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74250f34-9cea-457f-9cf3-75b9a212a04b None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:08.773824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74250f34-9cea-457f-9cf3-75b9a212a04b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:08.774573 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74250f34-9cea-457f-9cf3-75b9a212a04b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:08.776202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74250f34-9cea-457f-9cf3-75b9a212a04b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:08.781863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 451/917] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Aug 30 14:39:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:08.787360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca68264-b761-431f-96cc-21dbc717f8c5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:08.840951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:08.894585 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca68264-b761-431f-96cc-21dbc717f8c5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:08.917859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceff4e5-71e6-4a7b-9328-c6fd3cfc3814 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:08.918198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceff4e5-71e6-4a7b-9328-c6fd3cfc3814 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:08.918416 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceff4e5-71e6-4a7b-9328-c6fd3cfc3814 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:08.920028 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 452/918] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:39:08.924914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff4a29-9d05-459a-a50d-5544b92c8747 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:08.925193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff4a29-9d05-459a-a50d-5544b92c8747 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:08.925465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff4a29-9d05-459a-a50d-5544b92c8747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:08.941150 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51711d24-7dfd-417f-88bb-ecb114018a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:08.942063 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51711d24-7dfd-417f-88bb-ecb114018a65 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:08.942542 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51711d24-7dfd-417f-88bb-ecb114018a65 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:08.942965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51711d24-7dfd-417f-88bb-ecb114018a65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:08.943970 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51711d24-7dfd-417f-88bb-ecb114018a65 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:08.944639 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-4cff4a29-9d05-459a-a50d-5544b92c8747 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:08.945891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-4cff4a29-9d05-459a-a50d-5544b92c8747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:08.946087 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51711d24-7dfd-417f-88bb-ecb114018a65 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:09.017391 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 453/919] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:39:09.033878 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 467/920] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:39:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:39:09.298583 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff4a29-9d05-459a-a50d-5544b92c8747 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:09.362843 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:09.404843 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff4a29-9d05-459a-a50d-5544b92c8747 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:09.454010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc659b-2fe4-496d-8110-e3c29629451b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:09.454643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc659b-2fe4-496d-8110-e3c29629451b None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:09.454920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc659b-2fe4-496d-8110-e3c29629451b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:09.455228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc659b-2fe4-496d-8110-e3c29629451b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:09.455808 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc659b-2fe4-496d-8110-e3c29629451b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:09.457177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc659b-2fe4-496d-8110-e3c29629451b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-8cdc659b-2fe4-496d-8110-e3c29629451b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:09.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:09.470294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:09.470294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:09.470294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:09.470294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:09.470294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:09.470294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:09.470294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:39:09.470294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:39:09.470294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:39:09.470294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:39:09.470294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:09.470294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:09.470294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:09.470294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:09.470294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:09.470294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:09.472304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:09.472304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:09.472304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:09.472304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:09.472304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:09.472304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:09.472304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:09.472304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:09.472304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:09.472304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:09.472304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:39:09.472304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:09.472304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:39:09.472304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:09.472304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 454/921] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 30 14:39:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:09.479691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51711d24-7dfd-417f-88bb-ecb114018a65 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:09.595853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51711d24-7dfd-417f-88bb-ecb114018a65 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:09.603856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:09.633563 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5022c0-e190-4eff-8fdc-b1c38f1d05c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:09.634413 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5022c0-e190-4eff-8fdc-b1c38f1d05c4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:09.634754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5022c0-e190-4eff-8fdc-b1c38f1d05c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:09.635204 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5022c0-e190-4eff-8fdc-b1c38f1d05c4 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:09.636294 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5022c0-e190-4eff-8fdc-b1c38f1d05c4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:09.636594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5022c0-e190-4eff-8fdc-b1c38f1d05c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:09.638014 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5022c0-e190-4eff-8fdc-b1c38f1d05c4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:09.643287 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 468/922] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 30 14:39:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:09.651211 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc659b-2fe4-496d-8110-e3c29629451b None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:09.764127 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc659b-2fe4-496d-8110-e3c29629451b None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:09.787367 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:09.799617 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17379b-2168-43e8-be4e-0ef835d563da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:09.800216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17379b-2168-43e8-be4e-0ef835d563da None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:09.814418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17379b-2168-43e8-be4e-0ef835d563da None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:09.814418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17379b-2168-43e8-be4e-0ef835d563da None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:09.814418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17379b-2168-43e8-be4e-0ef835d563da None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:09.814418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17379b-2168-43e8-be4e-0ef835d563da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:09.814418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17379b-2168-43e8-be4e-0ef835d563da None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:09.814418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 455/923] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 30 14:39:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:09.816034 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5022c0-e190-4eff-8fdc-b1c38f1d05c4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:09.922077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5022c0-e190-4eff-8fdc-b1c38f1d05c4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:09.954392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334b412-9362-4290-a6bf-f660f99aafc6 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:09.954696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334b412-9362-4290-a6bf-f660f99aafc6 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:09.954949 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334b412-9362-4290-a6bf-f660f99aafc6 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:09.956436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 456/924] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:39:09.961439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4ee5a-3750-47d6-8cfc-731a79864896 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:09.961723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4ee5a-3750-47d6-8cfc-731a79864896 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:09.961998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4ee5a-3750-47d6-8cfc-731a79864896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:09.964296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d6485-a340-46bc-a6ce-1d5abd46375b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:09.965191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d6485-a340-46bc-a6ce-1d5abd46375b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:09.965677 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d6485-a340-46bc-a6ce-1d5abd46375b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:09.966152 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d6485-a340-46bc-a6ce-1d5abd46375b None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:09.967083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d6485-a340-46bc-a6ce-1d5abd46375b None admin] RBAC: Adding query filter params (interface=public, service_id=8734061efad0467092aad94c16e9ba96, region_id=RegionOne) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:09.967479 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d6485-a340-46bc-a6ce-1d5abd46375b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:09.968967 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d6485-a340-46bc-a6ce-1d5abd46375b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:09.974833 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 469/925] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Aug 30 14:39:09 2022] GET /identity/v3/endpoints?service_id=8734061efad0467092aad94c16e9ba96&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:09.980498 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d6485-a340-46bc-a6ce-1d5abd46375b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:09.982816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-a9a4ee5a-3750-47d6-8cfc-731a79864896 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:09.984504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a9a4ee5a-3750-47d6-8cfc-731a79864896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:10.029699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:10.063388 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 457/926] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:39:10.065642 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b77f-b0d3-4d11-94d3-1521832fa629 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:10.066028 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b77f-b0d3-4d11-94d3-1521832fa629 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:10.066264 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b77f-b0d3-4d11-94d3-1521832fa629 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:10.067829 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 458/927] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:10.072558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdcb08-4415-4b8c-8b2c-a14d24c25b40 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:10.072965 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdcb08-4415-4b8c-8b2c-a14d24c25b40 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:10.073331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdcb08-4415-4b8c-8b2c-a14d24c25b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:10.093091 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-47fdcb08-4415-4b8c-8b2c-a14d24c25b40 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:10.094411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-47fdcb08-4415-4b8c-8b2c-a14d24c25b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:10.106997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d6485-a340-46bc-a6ce-1d5abd46375b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:10.147782 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565b9fa-a6d5-4d44-9eaa-9a491730f9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:10.148556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565b9fa-a6d5-4d44-9eaa-9a491730f9ef None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:10.149238 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565b9fa-a6d5-4d44-9eaa-9a491730f9ef None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:10.149605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565b9fa-a6d5-4d44-9eaa-9a491730f9ef None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:10.151390 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565b9fa-a6d5-4d44-9eaa-9a491730f9ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:10.153192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565b9fa-a6d5-4d44-9eaa-9a491730f9ef None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:10.161138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 470/928] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 30 14:39:09 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:39:10.182971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 459/929] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:10.348114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565b9fa-a6d5-4d44-9eaa-9a491730f9ef None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:10.454920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:10.458046 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdcb08-4415-4b8c-8b2c-a14d24c25b40 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:10.458534 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565b9fa-a6d5-4d44-9eaa-9a491730f9ef None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:10.493965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793f45e-1ca9-4b68-8e1c-7319f9f0c537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:10.494697 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793f45e-1ca9-4b68-8e1c-7319f9f0c537 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:10.495057 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793f45e-1ca9-4b68-8e1c-7319f9f0c537 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:10.495433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793f45e-1ca9-4b68-8e1c-7319f9f0c537 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:10.496109 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793f45e-1ca9-4b68-8e1c-7319f9f0c537 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:10.497573 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793f45e-1ca9-4b68-8e1c-7319f9f0c537 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:10.506663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-9793f45e-1ca9-4b68-8e1c-7319f9f0c537 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:39:10.506663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:10.506663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:39:10.506663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:39:10.506663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:39:10.506663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:39:10.506663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:39:10.506663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:10.506663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:39:10.506663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:10.506663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:10.506663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:10.506663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:10.506663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:10.506663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:10.506663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:10.507832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:10.508963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:10.508963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:10.508963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:10.508963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:10.508963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:10.508963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:10.508963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:10.508963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:10.508963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:10.508963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:10.508963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:39:10.508963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:10.508963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:39:10.508963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:10.510891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 471/930] 10.222.0.27 () {56 vars in 1132 bytes} [Tue Aug 30 14:39:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:10.518164 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793f45e-1ca9-4b68-8e1c-7319f9f0c537 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:10.526468 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:10.602676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdcb08-4415-4b8c-8b2c-a14d24c25b40 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:10.625774 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793f45e-1ca9-4b68-8e1c-7319f9f0c537 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:10.649591 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee32a36-f10d-43e4-9077-fc8a116bcf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:10.651398 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee32a36-f10d-43e4-9077-fc8a116bcf90 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:10.651675 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee32a36-f10d-43e4-9077-fc8a116bcf90 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:10.652081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee32a36-f10d-43e4-9077-fc8a116bcf90 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:10.665935 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb572e-4715-44e5-a248-53edae0d0dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:10.666302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee32a36-f10d-43e4-9077-fc8a116bcf90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:39:10.666841 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb572e-4715-44e5-a248-53edae0d0dd8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:10.667289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb572e-4715-44e5-a248-53edae0d0dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:10.667661 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb572e-4715-44e5-a248-53edae0d0dd8 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:10.668408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb572e-4715-44e5-a248-53edae0d0dd8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:10.668816 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb572e-4715-44e5-a248-53edae0d0dd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:10.670289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb572e-4715-44e5-a248-53edae0d0dd8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:10.671063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-9ee32a36-f10d-43e4-9077-fc8a116bcf90 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:39:10.671063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:10.671063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:10.671063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:10.671063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:10.671063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:10.671063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:10.671063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:10.671063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:10.671063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:10.671063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:10.671063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:10.671063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:39:10.671063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:39:10.671063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:39:10.671063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:39:10.672850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:39:10.672850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:39:10.672850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:39:10.672850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:39:10.672850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:10.672850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:10.672850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:10.672850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:10.672850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:10.672850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:10.672850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:10.672850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:10.672850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:10.672850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:10.672850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:10.672850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:10.674043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:10.674043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:10.674043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:10.674043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:10.674043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:39:10.674043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:39:10.674043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:39:10.674043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:39:10.674043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:39:10.674043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:39:10.674043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:39:10.674043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:10.674043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 460/931] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Aug 30 14:39:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:10.680319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee32a36-f10d-43e4-9077-fc8a116bcf90 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:10.681620 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 472/932] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 30 14:39:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:10.687291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb572e-4715-44e5-a248-53edae0d0dd8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:10.721144 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:10.809244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb572e-4715-44e5-a248-53edae0d0dd8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:10.816459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee32a36-f10d-43e4-9077-fc8a116bcf90 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:10.847533 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:10.849648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5dca18-a803-4af7-8859-2339b5410141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:10.850533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5dca18-a803-4af7-8859-2339b5410141 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:10.850936 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5dca18-a803-4af7-8859-2339b5410141 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:10.851424 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5dca18-a803-4af7-8859-2339b5410141 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:10.852439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5dca18-a803-4af7-8859-2339b5410141 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:10.852940 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5dca18-a803-4af7-8859-2339b5410141 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:10.855224 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5dca18-a803-4af7-8859-2339b5410141 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:10.863044 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23284580-120e-4a3a-bdb0-eee4d09ab9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:10.863707 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 473/933] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 30 14:39:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:10.863823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23284580-120e-4a3a-bdb0-eee4d09ab9a3 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:10.864037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23284580-120e-4a3a-bdb0-eee4d09ab9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:10.864431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23284580-120e-4a3a-bdb0-eee4d09ab9a3 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:10.865101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23284580-120e-4a3a-bdb0-eee4d09ab9a3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:10.865400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23284580-120e-4a3a-bdb0-eee4d09ab9a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:10.866840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23284580-120e-4a3a-bdb0-eee4d09ab9a3 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:10.869975 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5dca18-a803-4af7-8859-2339b5410141 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:10.880577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 461/934] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Aug 30 14:39:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:10.883901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23284580-120e-4a3a-bdb0-eee4d09ab9a3 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:10.970450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:10.982142 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5dca18-a803-4af7-8859-2339b5410141 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:10.986820 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23284580-120e-4a3a-bdb0-eee4d09ab9a3 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:11.021001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18025fcd-2765-4322-88d8-9c27d33f815e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:11.021550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c2c00-82cc-43a5-9dfe-99bd47570879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:11.022180 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c2c00-82cc-43a5-9dfe-99bd47570879 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:11.022455 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c2c00-82cc-43a5-9dfe-99bd47570879 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:11.022681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18025fcd-2765-4322-88d8-9c27d33f815e None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:11.022780 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c2c00-82cc-43a5-9dfe-99bd47570879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:11.023356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18025fcd-2765-4322-88d8-9c27d33f815e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:11.024114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18025fcd-2765-4322-88d8-9c27d33f815e None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:11.025960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18025fcd-2765-4322-88d8-9c27d33f815e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:11.027694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18025fcd-2765-4322-88d8-9c27d33f815e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:11.035730 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c2c00-82cc-43a5-9dfe-99bd47570879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:11.037307 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c2c00-82cc-43a5-9dfe-99bd47570879 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:11.052336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-f83c2c00-82cc-43a5-9dfe-99bd47570879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:11.052336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:11.052336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:11.052336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:11.052336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:11.052336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:11.052336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:11.052336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:11.052336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:11.052336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:11.052336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:11.052336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:11.052336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:39:11.052336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:39:11.052336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:39:11.052336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:11.053323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:39:11.055701 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:39:11.055701 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:11.055701 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:11.055701 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 462/935] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:11.059809 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c2c00-82cc-43a5-9dfe-99bd47570879 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:11.084024 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:11.119679 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 474/936] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:39:10 2022] POST /identity/v3/endpoints => generated 347 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:39:11.178034 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c2c00-82cc-43a5-9dfe-99bd47570879 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:11.218749 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24082b-8cae-4837-9625-2cae0e783052 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:11.219195 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24082b-8cae-4837-9625-2cae0e783052 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:11.219582 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24082b-8cae-4837-9625-2cae0e783052 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:11.223234 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dac6dd-f38a-461b-b68b-6791d93cdfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:11.223906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dac6dd-f38a-461b-b68b-6791d93cdfe9 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:11.224186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dac6dd-f38a-461b-b68b-6791d93cdfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:11.224454 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dac6dd-f38a-461b-b68b-6791d93cdfe9 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:11.225110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dac6dd-f38a-461b-b68b-6791d93cdfe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:11.225407 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dac6dd-f38a-461b-b68b-6791d93cdfe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:11.226811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dac6dd-f38a-461b-b68b-6791d93cdfe9 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:11.227320 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 475/937] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:11 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:11.229446 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183d342-180d-4f2e-96fa-c839ec789c21 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:11.229875 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183d342-180d-4f2e-96fa-c839ec789c21 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:11.230305 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183d342-180d-4f2e-96fa-c839ec789c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:11.247069 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 463/938] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:11.253673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dac6dd-f38a-461b-b68b-6791d93cdfe9 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:11.255892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a183d342-180d-4f2e-96fa-c839ec789c21 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:11.257695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a183d342-180d-4f2e-96fa-c839ec789c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:11.355629 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 476/939] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:11.374578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:11.402246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dac6dd-f38a-461b-b68b-6791d93cdfe9 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:11.438583 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b0e3a-2388-4186-9163-be703d61c745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:11.439509 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b0e3a-2388-4186-9163-be703d61c745 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:11.439966 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b0e3a-2388-4186-9163-be703d61c745 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:11.440375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b0e3a-2388-4186-9163-be703d61c745 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:11.448814 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b0e3a-2388-4186-9163-be703d61c745 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:11.450395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b0e3a-2388-4186-9163-be703d61c745 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:11.461962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-0e9b0e3a-2388-4186-9163-be703d61c745 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:39:11.461962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:11.461962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:11.461962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:11.461962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:11.461962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:11.461962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:11.461962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:11.461962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:11.461962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:11.461962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:11.461962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:11.461962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:39:11.461962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:39:11.461962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:39:11.461962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:11.463148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:11.464333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:11.464333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:11.464333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:39:11.464333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:39:11.464333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:39:11.464333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:39:11.464333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:39:11.464333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:39:11.464333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:39:11.464333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:11.466442 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 464/940] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 30 14:39:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:11.472227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183d342-180d-4f2e-96fa-c839ec789c21 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:11.554102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:11.577409 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183d342-180d-4f2e-96fa-c839ec789c21 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:11.613287 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93823e29-f319-466c-909c-56860adf33f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:11.613902 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93823e29-f319-466c-909c-56860adf33f9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:11.614181 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93823e29-f319-466c-909c-56860adf33f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:11.614451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93823e29-f319-466c-909c-56860adf33f9 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:11.615246 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93823e29-f319-466c-909c-56860adf33f9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:11.615541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93823e29-f319-466c-909c-56860adf33f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:11.617044 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93823e29-f319-466c-909c-56860adf33f9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:11.635003 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 477/941] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Aug 30 14:39:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:11.641668 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b0e3a-2388-4186-9163-be703d61c745 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:11.749346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b0e3a-2388-4186-9163-be703d61c745 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:11.785390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11b54a-8c74-4b6f-8461-72e9fc9fad6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:11.786396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11b54a-8c74-4b6f-8461-72e9fc9fad6b None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:11.786924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11b54a-8c74-4b6f-8461-72e9fc9fad6b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:11.787412 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11b54a-8c74-4b6f-8461-72e9fc9fad6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:11.790626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:11.799238 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11b54a-8c74-4b6f-8461-72e9fc9fad6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:11.801035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11b54a-8c74-4b6f-8461-72e9fc9fad6b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:11.802464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93823e29-f319-466c-909c-56860adf33f9 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:11.816579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-dd11b54a-8c74-4b6f-8461-72e9fc9fad6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:11.816579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:11.816579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:11.816579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:11.816579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:11.816579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:11.816579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:11.816579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:11.816579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:11.816579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:11.816579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:11.816579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:11.816579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:39:11.816579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:39:11.816579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:39:11.816579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:11.818452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:39:11.819906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:39:11.819906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:11.819906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:11.822180 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 465/942] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:11.824268 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11b54a-8c74-4b6f-8461-72e9fc9fad6b None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:11.947761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11b54a-8c74-4b6f-8461-72e9fc9fad6b None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:11.970336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93823e29-f319-466c-909c-56860adf33f9 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:11.983085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd016ad7-5e8e-4c43-9bb2-763abadb4217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:11.983749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd016ad7-5e8e-4c43-9bb2-763abadb4217 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:11.984030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd016ad7-5e8e-4c43-9bb2-763abadb4217 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:11.984401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd016ad7-5e8e-4c43-9bb2-763abadb4217 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:11.985053 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd016ad7-5e8e-4c43-9bb2-763abadb4217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:11.985408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd016ad7-5e8e-4c43-9bb2-763abadb4217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:11.986807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd016ad7-5e8e-4c43-9bb2-763abadb4217 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:12.007170 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 466/943] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:12.008044 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd016ad7-5e8e-4c43-9bb2-763abadb4217 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:12.018156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bbc88-dfc5-4bde-8130-520e18566fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:12.018773 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bbc88-dfc5-4bde-8130-520e18566fcd None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:12.019055 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bbc88-dfc5-4bde-8130-520e18566fcd None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:12.019366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bbc88-dfc5-4bde-8130-520e18566fcd None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:12.020025 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4bbc88-dfc5-4bde-8130-520e18566fcd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:12.021985 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4bbc88-dfc5-4bde-8130-520e18566fcd None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:12.030586 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: WARNING keystone.server.flask.common [None req-7e4bbc88-dfc5-4bde-8130-520e18566fcd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 14:39:12.034637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:12.072716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:12.099404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.sql.core [None req-7e4bbc88-dfc5-4bde-8130-520e18566fcd None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 14:39:12.099404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 14:39:12.099404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [parameters: {'id': '1b7a53c5b4514e47a2dee00975ddf9b9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 14:39:12.099404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88059) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 14:39:12.122668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-7e4bbc88-dfc5-4bde-8130-520e18566fcd None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1b7a53c5b4514e47a2dee00975ddf9b9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1b7a53c5b4514e47a2dee00975ddf9b9. Aug 30 14:39:12.122668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:12.122668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:39:12.122668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:39:12.122668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:39:12.122668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:39:12.122668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:39:12.122668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:39:12.122668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:39:12.122668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:39:12.122668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:39:12.122668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:39:12.122668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result.read() Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 30 14:39:12.123751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application yield resource Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise self.value Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application session.commit() Aug 30 14:39:12.124926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.session.flush() Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application compat.raise_( Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 14:39:12.126009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 14:39:12.127225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 14:39:12.127225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 14:39:12.127225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 14:39:12.127225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 14:39:12.127225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 14:39:12.127225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 14:39:12.127225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 14:39:12.127225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 14:39:12.127225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 14:39:12.127225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 14:39:12.127225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 14:39:12.127225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 14:39:12.127225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 14:39:12.127225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 14:39:12.127225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result.read() Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:39:12.128371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:39:12.130012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:39:12.130012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:39:12.130012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:39:12.130012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:39:12.130012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:39:12.130012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:39:12.130012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 14:39:12.130012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 14:39:12.130012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [parameters: {'id': '1b7a53c5b4514e47a2dee00975ddf9b9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 14:39:12.130012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 14:39:12.130012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:12.130012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:12.130012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:12.130012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:12.130012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:12.132203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:12.139463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:12.139463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 30 14:39:12.139463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 30 14:39:12.139463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:12.139463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:12.139463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 30 14:39:12.139463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 30 14:39:12.139463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1b7a53c5b4514e47a2dee00975ddf9b9. Aug 30 14:39:12.139463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:12.139463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 478/944] 10.222.0.27 () {60 vars in 1140 bytes} [Tue Aug 30 14:39:11 2022] POST /identity/v3/projects => generated 262 bytes in 329 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:12.139463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bbc88-dfc5-4bde-8130-520e18566fcd None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:12.140234 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd016ad7-5e8e-4c43-9bb2-763abadb4217 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:12.177265 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862d9ec-f866-42f8-88ec-b85eff7d665e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:12.178750 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862d9ec-f866-42f8-88ec-b85eff7d665e None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:12.179042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862d9ec-f866-42f8-88ec-b85eff7d665e None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:12.179329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862d9ec-f866-42f8-88ec-b85eff7d665e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:12.187673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862d9ec-f866-42f8-88ec-b85eff7d665e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:12.189105 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862d9ec-f866-42f8-88ec-b85eff7d665e None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:12.201087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-2862d9ec-f866-42f8-88ec-b85eff7d665e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:12.201087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:12.201087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:12.201087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:12.201087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:12.201087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:12.201087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:12.201087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:12.201087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:12.201087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:12.201087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:12.201087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:12.201087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:39:12.201087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:39:12.201087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:12.202006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:12.202942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:39:12.202942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:39:12.202942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:39:12.202942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:39:12.202942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:39:12.202942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:39:12.202942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:12.202942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:12.202942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 467/945] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:39:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:12.204011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b4ab5-3cb5-490f-a95b-30ad8ae4df23 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:12.204011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b4ab5-3cb5-490f-a95b-30ad8ae4df23 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:12.204239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b4ab5-3cb5-490f-a95b-30ad8ae4df23 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:12.205518 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 468/946] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:12.210074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b4ab5-3cb5-490f-a95b-30ad8ae4df23 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:12.255138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bbc88-dfc5-4bde-8130-520e18566fcd None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:12.293120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b120723-0c66-43d8-8a15-d1a3177e630f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:12.293755 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b120723-0c66-43d8-8a15-d1a3177e630f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:12.294028 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b120723-0c66-43d8-8a15-d1a3177e630f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:12.294292 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b120723-0c66-43d8-8a15-d1a3177e630f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:12.301385 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b120723-0c66-43d8-8a15-d1a3177e630f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:12.303043 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b120723-0c66-43d8-8a15-d1a3177e630f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:12.314738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-2b120723-0c66-43d8-8a15-d1a3177e630f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:12.314738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:12.314738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:12.314738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:12.314738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:12.314738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:12.314738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:12.314738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:12.314738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:12.314738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:12.314738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:12.314738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:12.314738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:39:12.314738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:39:12.314738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:12.315849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:12.316795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:39:12.316795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:39:12.316795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:39:12.316795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:39:12.316795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:39:12.316795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:39:12.316795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:12.316795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:12.316795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 479/947] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:39:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:12.317297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eea1b-b6e5-4ab9-b220-cd9f000f7a9e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:12.317982 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eea1b-b6e5-4ab9-b220-cd9f000f7a9e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:12.317982 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eea1b-b6e5-4ab9-b220-cd9f000f7a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:12.329186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b4ab5-3cb5-490f-a95b-30ad8ae4df23 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:12.338501 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:12.342940 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-790eea1b-b6e5-4ab9-b220-cd9f000f7a9e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:12.344776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-790eea1b-b6e5-4ab9-b220-cd9f000f7a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:12.369841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56cbfc-6cec-41ff-85a2-984f30b781e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:12.369841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56cbfc-6cec-41ff-85a2-984f30b781e7 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:12.370569 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56cbfc-6cec-41ff-85a2-984f30b781e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:12.370853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56cbfc-6cec-41ff-85a2-984f30b781e7 None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:12.371618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56cbfc-6cec-41ff-85a2-984f30b781e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:12.371930 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56cbfc-6cec-41ff-85a2-984f30b781e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:12.373408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56cbfc-6cec-41ff-85a2-984f30b781e7 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:12.388314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 469/948] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:39:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:12.389172 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56cbfc-6cec-41ff-85a2-984f30b781e7 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:12.405192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:12.435136 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 480/949] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:12.435136 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eea1b-b6e5-4ab9-b220-cd9f000f7a9e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:12.531218 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56cbfc-6cec-41ff-85a2-984f30b781e7 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:12.570400 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eea1b-b6e5-4ab9-b220-cd9f000f7a9e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:12.587555 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918d448-9e11-41a6-a100-371693895ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:12.589107 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918d448-9e11-41a6-a100-371693895ab0 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:12.589857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918d448-9e11-41a6-a100-371693895ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:12.590357 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918d448-9e11-41a6-a100-371693895ab0 None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:12.591447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918d448-9e11-41a6-a100-371693895ab0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:12.592028 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918d448-9e11-41a6-a100-371693895ab0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:12.595092 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918d448-9e11-41a6-a100-371693895ab0 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:12.610565 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f27e0f-cffe-4dd5-8257-4076aa17b9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:12.611315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f27e0f-cffe-4dd5-8257-4076aa17b9e2 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:12.611593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f27e0f-cffe-4dd5-8257-4076aa17b9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:12.611861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f27e0f-cffe-4dd5-8257-4076aa17b9e2 None admin] PATH_INFO: `/v3/projects/1147574d59c7404baf76a24149684f6e/users/912e9c732f194fa496a9111bbfab3efa/roles/bcecc24215984de9a00652caa5433ffe` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:12.614776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f27e0f-cffe-4dd5-8257-4076aa17b9e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=1147574d59c7404baf76a24149684f6e, user_id=912e9c732f194fa496a9111bbfab3efa, role_id=bcecc24215984de9a00652caa5433ffe)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:12.616242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f27e0f-cffe-4dd5-8257-4076aa17b9e2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:12.621769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 470/950] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 30 14:39:12 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:12.711275 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 481/951] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:39:12 2022] PUT /identity/v3/projects/1147574d59c7404baf76a24149684f6e/users/912e9c732f194fa496a9111bbfab3efa/roles/bcecc24215984de9a00652caa5433ffe => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:39:12.745918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918d448-9e11-41a6-a100-371693895ab0 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:12.783772 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:12.873361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918d448-9e11-41a6-a100-371693895ab0 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:12.909229 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e6306-b21f-479c-b386-4e4a16b581d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:12.909858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e6306-b21f-479c-b386-4e4a16b581d3 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:12.910131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e6306-b21f-479c-b386-4e4a16b581d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:12.910400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e6306-b21f-479c-b386-4e4a16b581d3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:12.910981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e6306-b21f-479c-b386-4e4a16b581d3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:12.912374 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e6306-b21f-479c-b386-4e4a16b581d3 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-6e1e6306-b21f-479c-b386-4e4a16b581d3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:12.922124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:12.923562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:12.923562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:12.923562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:12.923562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:12.923562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:12.923562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:12.923562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:39:12.923562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:39:12.923562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:39:12.923562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:39:12.923562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:12.923562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:12.923562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:12.923562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:12.923562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:12.923562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:12.924811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:12.924811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:12.924811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:12.924811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:12.924811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:12.924811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:12.924811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:12.924811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:12.924811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:12.924811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:12.924811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:39:12.924811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:12.924811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:39:12.924811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:12.924811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 471/952] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 30 14:39:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:12.929570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f27e0f-cffe-4dd5-8257-4076aa17b9e2 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:12.978394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:13.039651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f27e0f-cffe-4dd5-8257-4076aa17b9e2 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:13.081717 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4848a-bab2-4f67-ae36-1763b0631ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:13.082334 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4848a-bab2-4f67-ae36-1763b0631ebb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:13.082615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4848a-bab2-4f67-ae36-1763b0631ebb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:13.082893 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4848a-bab2-4f67-ae36-1763b0631ebb None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:13.083556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4848a-bab2-4f67-ae36-1763b0631ebb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:13.083862 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4848a-bab2-4f67-ae36-1763b0631ebb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:13.085260 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4848a-bab2-4f67-ae36-1763b0631ebb None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:13.093963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 482/953] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 30 14:39:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:13.099693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e6306-b21f-479c-b386-4e4a16b581d3 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:13.196996 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:13.206001 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e6306-b21f-479c-b386-4e4a16b581d3 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:13.240146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac9296-878d-4f6c-b6a6-be76bbadf258 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:13.240428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac9296-878d-4f6c-b6a6-be76bbadf258 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:13.240700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac9296-878d-4f6c-b6a6-be76bbadf258 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:13.242327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 483/954] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:13.246313 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15351f5a-aeb2-4bd8-9813-cfc5b055aff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:13.247044 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15351f5a-aeb2-4bd8-9813-cfc5b055aff6 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:13.247380 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e2e3b-9c2c-4c1e-94b1-179cd5fb01fd None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:13.247494 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15351f5a-aeb2-4bd8-9813-cfc5b055aff6 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:13.247669 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e2e3b-9c2c-4c1e-94b1-179cd5fb01fd None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:13.247942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15351f5a-aeb2-4bd8-9813-cfc5b055aff6 None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:13.248060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e2e3b-9c2c-4c1e-94b1-179cd5fb01fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:13.248742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15351f5a-aeb2-4bd8-9813-cfc5b055aff6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:13.249135 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15351f5a-aeb2-4bd8-9813-cfc5b055aff6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:13.250581 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15351f5a-aeb2-4bd8-9813-cfc5b055aff6 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:13.256557 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 472/955] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 30 14:39:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:13.262808 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15351f5a-aeb2-4bd8-9813-cfc5b055aff6 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:13.279546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c47e2e3b-9c2c-4c1e-94b1-179cd5fb01fd None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:13.281491 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c47e2e3b-9c2c-4c1e-94b1-179cd5fb01fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:13.359250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 484/956] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:39:13.379799 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15351f5a-aeb2-4bd8-9813-cfc5b055aff6 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:13.417716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69aef4-aa87-4c0f-85f5-927eb4a66e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:13.418577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69aef4-aa87-4c0f-85f5-927eb4a66e4b None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:13.419022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69aef4-aa87-4c0f-85f5-927eb4a66e4b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:13.419462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69aef4-aa87-4c0f-85f5-927eb4a66e4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:13.420294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69aef4-aa87-4c0f-85f5-927eb4a66e4b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:13.421894 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69aef4-aa87-4c0f-85f5-927eb4a66e4b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:13.501496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 473/957] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:39:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:39:13.651989 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e2e3b-9c2c-4c1e-94b1-179cd5fb01fd None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:13.688767 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:13.760132 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e2e3b-9c2c-4c1e-94b1-179cd5fb01fd None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:13.802915 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79bafa-ca24-41f9-9270-376cd1523d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:13.803818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79bafa-ca24-41f9-9270-376cd1523d45 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:13.804248 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79bafa-ca24-41f9-9270-376cd1523d45 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:13.804690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79bafa-ca24-41f9-9270-376cd1523d45 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:13.805437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79bafa-ca24-41f9-9270-376cd1523d45 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:13.806999 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79bafa-ca24-41f9-9270-376cd1523d45 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-be79bafa-ca24-41f9-9270-376cd1523d45 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:13.816226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:13.817569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:13.817569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:13.817569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:13.817569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:13.817569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:13.817569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:13.817569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:39:13.817569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:39:13.817569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:39:13.817569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:39:13.817569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:13.817569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:13.817569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:13.817569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:13.817569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:13.817569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:13.818652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:13.818652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:13.818652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:13.818652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:13.818652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:13.818652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:13.818652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:13.818652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:13.818652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:13.818652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:13.818652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:39:13.818652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:13.818652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:39:13.818652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:13.821216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 485/958] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:39:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:13.827120 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69aef4-aa87-4c0f-85f5-927eb4a66e4b None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:13.847203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:13.940517 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69aef4-aa87-4c0f-85f5-927eb4a66e4b None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:13.978372 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8d527-25ed-46d6-91a5-b655a014786e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:13.978980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8d527-25ed-46d6-91a5-b655a014786e None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:13.979288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8d527-25ed-46d6-91a5-b655a014786e None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:13.979563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8d527-25ed-46d6-91a5-b655a014786e None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:13.980196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8d527-25ed-46d6-91a5-b655a014786e None admin] RBAC: Adding query filter params (name=compute) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:13.980496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8d527-25ed-46d6-91a5-b655a014786e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:13.981922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8d527-25ed-46d6-91a5-b655a014786e None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:13.987240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 474/959] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 30 14:39:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:13.992852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79bafa-ca24-41f9-9270-376cd1523d45 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:14.101678 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:14.108592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79bafa-ca24-41f9-9270-376cd1523d45 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:14.143076 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff84fd-0415-45d7-a6a9-201c0f2661e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:14.143715 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff84fd-0415-45d7-a6a9-201c0f2661e4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:14.143972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff84fd-0415-45d7-a6a9-201c0f2661e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:14.144250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff84fd-0415-45d7-a6a9-201c0f2661e4 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:14.144881 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff84fd-0415-45d7-a6a9-201c0f2661e4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:14.145183 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff84fd-0415-45d7-a6a9-201c0f2661e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:14.146555 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff84fd-0415-45d7-a6a9-201c0f2661e4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:14.152218 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 486/960] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 30 14:39:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:14.727995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409aa4d4-a11e-4d54-8e48-dd4d89f87bef None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:14.728395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409aa4d4-a11e-4d54-8e48-dd4d89f87bef None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:14.728591 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409aa4d4-a11e-4d54-8e48-dd4d89f87bef None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:14.730227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 475/961] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:14.735477 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6bb16-b523-4551-a66f-3b0515ca9f91 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:14.735913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6bb16-b523-4551-a66f-3b0515ca9f91 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:14.736027 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6bb16-b523-4551-a66f-3b0515ca9f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:14.750941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d5c31-cd83-498b-952a-8b7c55e38c40 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:14.751431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d5c31-cd83-498b-952a-8b7c55e38c40 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:14.751870 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d5c31-cd83-498b-952a-8b7c55e38c40 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:14.754640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 476/962] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:14.755771 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-8be6bb16-b523-4551-a66f-3b0515ca9f91 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:14.756962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8be6bb16-b523-4551-a66f-3b0515ca9f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:14.762537 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55ef0d-e231-4c40-9416-4275c1caed3c None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:14.762824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55ef0d-e231-4c40-9416-4275c1caed3c None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:14.763121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55ef0d-e231-4c40-9416-4275c1caed3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:14.781454 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-9c55ef0d-e231-4c40-9416-4275c1caed3c None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:14.782646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9c55ef0d-e231-4c40-9416-4275c1caed3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:14.843589 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 487/963] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:14.862217 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 477/964] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:39:15.114094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6bb16-b523-4551-a66f-3b0515ca9f91 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:15.136101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55ef0d-e231-4c40-9416-4275c1caed3c None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:15.144913 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:15.205110 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:15.217713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6bb16-b523-4551-a66f-3b0515ca9f91 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:15.253877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55ef0d-e231-4c40-9416-4275c1caed3c None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:15.260014 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76109b53-5132-4294-b756-904a9829b117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:15.260634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76109b53-5132-4294-b756-904a9829b117 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:15.260906 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76109b53-5132-4294-b756-904a9829b117 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:15.261177 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76109b53-5132-4294-b756-904a9829b117 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:15.261840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76109b53-5132-4294-b756-904a9829b117 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:15.263290 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76109b53-5132-4294-b756-904a9829b117 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:15.296174 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6265be-c005-4895-b590-40a60e68a923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:15.296805 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6265be-c005-4895-b590-40a60e68a923 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:15.297081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6265be-c005-4895-b590-40a60e68a923 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:15.297350 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6265be-c005-4895-b590-40a60e68a923 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:15.308412 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6265be-c005-4895-b590-40a60e68a923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:39:15.313315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-3b6265be-c005-4895-b590-40a60e68a923 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:39:15.313315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:15.313315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:15.313315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:15.313315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:15.313315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:15.313315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:15.313315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:15.313315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:15.313315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:15.313315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:15.313315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:15.313315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:39:15.313315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:39:15.313315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:39:15.313315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:39:15.314287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:39:15.314287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:39:15.314287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:39:15.314287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:39:15.314287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:15.314287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:15.314287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:15.314287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:15.314287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:15.314287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:15.314287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:15.314287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:15.314287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:15.314287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:15.314287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:15.314287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:15.315742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:15.315742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:15.315742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:15.315742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:15.315742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:39:15.315742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:39:15.315742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:39:15.315742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:39:15.315742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:39:15.315742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:39:15.315742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:39:15.315742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:15.316661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 478/965] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Aug 30 14:39:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:15.320169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6265be-c005-4895-b590-40a60e68a923 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:15.350444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.sql.core [None req-76109b53-5132-4294-b756-904a9829b117 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 14:39:15.350444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 14:39:15.350444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [parameters: {'user_id': '5b5108f0e0ac451eb7a38ab104dffd63', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 14:39:15.350444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88059) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 14:39:15.357867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-76109b53-5132-4294-b756-904a9829b117 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 14:39:15.357867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:15.357867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:39:15.357867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:39:15.357867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:39:15.357867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:39:15.357867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:39:15.357867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:39:15.357867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:39:15.357867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:39:15.357867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:39:15.357867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:39:15.357867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:39:15.357867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result.read() Aug 30 14:39:15.357867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:39:15.358778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application yield resource Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise self.value Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application session.commit() Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 14:39:15.359852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.session.flush() Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application compat.raise_( Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 14:39:15.360781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 14:39:15.361740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 14:39:15.361740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 14:39:15.361740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 30 14:39:15.361740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 30 14:39:15.361740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 14:39:15.361740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 14:39:15.361740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 14:39:15.361740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 14:39:15.361740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 14:39:15.361740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 14:39:15.361740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 14:39:15.361740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 14:39:15.361740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 14:39:15.361740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 14:39:15.361740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:39:15.361740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception Aug 30 14:39:15.362621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:39:15.362621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:39:15.362621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:39:15.362621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:39:15.362621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:39:15.362621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:39:15.362621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:39:15.362621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:39:15.362621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:39:15.362621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:39:15.362621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:39:15.362621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application result.read() Aug 30 14:39:15.362621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:39:15.362621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:39:15.362621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:39:15.362621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:39:15.365920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:39:15.365920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:39:15.365920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:39:15.365920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:39:15.365920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 14:39:15.365920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 14:39:15.365920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [parameters: {'user_id': '5b5108f0e0ac451eb7a38ab104dffd63', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 14:39:15.365920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 14:39:15.365920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:15.365920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:15.365920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:15.365920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:15.365920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:15.365920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:15.365920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 30 14:39:15.368112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 30 14:39:15.370594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 30 14:39:15.370594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 30 14:39:15.370594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 14:39:15.370594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 14:39:15.370594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 14:39:15.370594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:15.370594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 488/966] 10.222.0.27 () {60 vars in 1131 bytes} [Tue Aug 30 14:39:15 2022] POST /identity/v3/users => generated 160 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:15.370594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76109b53-5132-4294-b756-904a9829b117 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:15.421693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:15.467700 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6265be-c005-4895-b590-40a60e68a923 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:15.480395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76109b53-5132-4294-b756-904a9829b117 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:15.512609 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28196e2b-e502-4a3e-8664-17feaf5e35ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:15.513765 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28196e2b-e502-4a3e-8664-17feaf5e35ed None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:15.514212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28196e2b-e502-4a3e-8664-17feaf5e35ed None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:15.514644 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28196e2b-e502-4a3e-8664-17feaf5e35ed None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:15.515555 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28196e2b-e502-4a3e-8664-17feaf5e35ed None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:15.516006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28196e2b-e502-4a3e-8664-17feaf5e35ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:15.517547 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28196e2b-e502-4a3e-8664-17feaf5e35ed None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:15.517888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9a65a-b396-47c5-afa0-bf3bede7894f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:15.518624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9a65a-b396-47c5-afa0-bf3bede7894f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:15.519001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9a65a-b396-47c5-afa0-bf3bede7894f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:15.519406 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9a65a-b396-47c5-afa0-bf3bede7894f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:15.527496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 479/967] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Aug 30 14:39:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:15.534898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9a65a-b396-47c5-afa0-bf3bede7894f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:15.535070 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc1a22-33d9-4833-bec9-4f919d900f72 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:15.535070 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc1a22-33d9-4833-bec9-4f919d900f72 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:15.535070 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc1a22-33d9-4833-bec9-4f919d900f72 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:15.535374 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9a65a-b396-47c5-afa0-bf3bede7894f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:15.535481 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 480/968] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:15.536011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc1a22-33d9-4833-bec9-4f919d900f72 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:15.543242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-afa9a65a-b396-47c5-afa0-bf3bede7894f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:39:15.543242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:15.543242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:15.543242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:15.543242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:15.543242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:15.543242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:15.543242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:15.543242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:15.543242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:15.543242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:15.543242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:15.543242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:39:15.543242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:39:15.543242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:39:15.543242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:15.544351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:15.545527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:15.545527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:15.545527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:39:15.545527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:39:15.545527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:39:15.545527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:39:15.545527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:39:15.545527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:39:15.545527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:39:15.545527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:15.551673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 489/969] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:39:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:15.551673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bccf88-2ab4-475d-9bfa-0901c42dde0f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:15.551673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bccf88-2ab4-475d-9bfa-0901c42dde0f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:15.551673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bccf88-2ab4-475d-9bfa-0901c42dde0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:15.568654 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-79bccf88-2ab4-475d-9bfa-0901c42dde0f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:15.570191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-79bccf88-2ab4-475d-9bfa-0901c42dde0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:15.574744 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:15.640281 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 490/970] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:15.643451 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc1a22-33d9-4833-bec9-4f919d900f72 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:15.644052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bccf88-2ab4-475d-9bfa-0901c42dde0f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:15.673561 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:15.682661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebcc39-07e7-488c-a919-3498bd4fcf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:15.683624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebcc39-07e7-488c-a919-3498bd4fcf09 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:15.686880 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebcc39-07e7-488c-a919-3498bd4fcf09 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:15.686880 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebcc39-07e7-488c-a919-3498bd4fcf09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:15.695372 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebcc39-07e7-488c-a919-3498bd4fcf09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:15.697721 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebcc39-07e7-488c-a919-3498bd4fcf09 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:15.715010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-25ebcc39-07e7-488c-a919-3498bd4fcf09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:15.715010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:15.715010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:15.715010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:15.715010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:15.715010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:15.715010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:15.715010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:15.715010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:15.715010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:15.715010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:15.715010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:15.715010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:39:15.715010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:39:15.715010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:39:15.715010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:15.716021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:39:15.716941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:39:15.716941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:15.716941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:15.719981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 481/971] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:15.725974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebcc39-07e7-488c-a919-3498bd4fcf09 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:15.768778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bccf88-2ab4-475d-9bfa-0901c42dde0f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:15.806105 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f7564-c8ae-454c-af0a-9497573e8c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:15.807093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f7564-c8ae-454c-af0a-9497573e8c93 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:15.807611 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f7564-c8ae-454c-af0a-9497573e8c93 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:15.808098 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f7564-c8ae-454c-af0a-9497573e8c93 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:15.809043 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f7564-c8ae-454c-af0a-9497573e8c93 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:15.809558 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f7564-c8ae-454c-af0a-9497573e8c93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:15.811338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f7564-c8ae-454c-af0a-9497573e8c93 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:15.837715 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:15.852683 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 491/972] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Aug 30 14:39:15 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:15.863426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebcc39-07e7-488c-a919-3498bd4fcf09 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:15.905044 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe04d2-c1c9-4480-bb41-bfc626420a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:15.905940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe04d2-c1c9-4480-bb41-bfc626420a07 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:15.906368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe04d2-c1c9-4480-bb41-bfc626420a07 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:15.906798 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe04d2-c1c9-4480-bb41-bfc626420a07 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:15.907664 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe04d2-c1c9-4480-bb41-bfc626420a07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:15.908142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe04d2-c1c9-4480-bb41-bfc626420a07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:15.909738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe04d2-c1c9-4480-bb41-bfc626420a07 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:15.915174 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f7564-c8ae-454c-af0a-9497573e8c93 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:15.931645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 482/973] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:15.935614 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe04d2-c1c9-4480-bb41-bfc626420a07 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:15.999833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:16.037797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f7564-c8ae-454c-af0a-9497573e8c93 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:16.083089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe04d2-c1c9-4480-bb41-bfc626420a07 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:16.085017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29a91b-b206-4bc8-89b8-8411c80b36b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:16.085661 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29a91b-b206-4bc8-89b8-8411c80b36b1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:16.085937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29a91b-b206-4bc8-89b8-8411c80b36b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:16.086206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29a91b-b206-4bc8-89b8-8411c80b36b1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:16.086791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29a91b-b206-4bc8-89b8-8411c80b36b1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:16.088964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29a91b-b206-4bc8-89b8-8411c80b36b1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-5d29a91b-b206-4bc8-89b8-8411c80b36b1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:16.102713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:16.104012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:16.104012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:16.104012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:16.104012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:16.104012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:16.104012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:16.104012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:39:16.104012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:39:16.104012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:39:16.104012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:39:16.104012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:16.104012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:16.104012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:16.104012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:16.104012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:16.104012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:16.104856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:16.104856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:16.104856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:16.104856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:16.104856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:16.104856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:16.104856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:16.104856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:16.104856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:16.104856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:16.104856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:39:16.104856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:16.104856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:39:16.104856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:16.104856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 492/974] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 30 14:39:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:16.105670 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb3455-45bd-42b2-9fc1-e411887ef7f3 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:16.105670 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb3455-45bd-42b2-9fc1-e411887ef7f3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:16.105873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb3455-45bd-42b2-9fc1-e411887ef7f3 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:16.107271 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 493/975] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:16.111356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb3455-45bd-42b2-9fc1-e411887ef7f3 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:16.128202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79ed02-f048-4d79-ac65-3803900edccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:16.129003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79ed02-f048-4d79-ac65-3803900edccc None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:16.129276 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79ed02-f048-4d79-ac65-3803900edccc None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:16.129934 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79ed02-f048-4d79-ac65-3803900edccc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:16.138589 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79ed02-f048-4d79-ac65-3803900edccc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:16.140094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79ed02-f048-4d79-ac65-3803900edccc None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:16.152928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-af79ed02-f048-4d79-ac65-3803900edccc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:39:16.152928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:16.152928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:16.152928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:16.152928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:16.152928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:16.152928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:16.152928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:16.152928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:16.152928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:16.152928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:16.152928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:16.152928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:39:16.152928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:39:16.152928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:39:16.152928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:16.154041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:16.155085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:16.155085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:16.155085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:39:16.155085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:39:16.155085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:39:16.155085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:39:16.155085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:39:16.155085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:39:16.155085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:39:16.155085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:16.155085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 483/976] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 30 14:39:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:16.159870 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a27dc-a673-4f55-829d-cf71cdd99df9 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:16.160154 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a27dc-a673-4f55-829d-cf71cdd99df9 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:16.160426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a27dc-a673-4f55-829d-cf71cdd99df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:16.177808 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-0b0a27dc-a673-4f55-829d-cf71cdd99df9 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:16.178075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:16.178992 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0b0a27dc-a673-4f55-829d-cf71cdd99df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:16.217905 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb3455-45bd-42b2-9fc1-e411887ef7f3 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:16.245088 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 484/977] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:16.245933 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a27dc-a673-4f55-829d-cf71cdd99df9 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:16.255901 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0c4a1-b3a3-4428-9f98-881a6f16c365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:16.256510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0c4a1-b3a3-4428-9f98-881a6f16c365 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:16.257009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0c4a1-b3a3-4428-9f98-881a6f16c365 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:16.257116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0c4a1-b3a3-4428-9f98-881a6f16c365 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:16.257684 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0c4a1-b3a3-4428-9f98-881a6f16c365 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:16.257987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0c4a1-b3a3-4428-9f98-881a6f16c365 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:16.259994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0c4a1-b3a3-4428-9f98-881a6f16c365 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:16.269310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 494/978] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 30 14:39:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:16.270384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0c4a1-b3a3-4428-9f98-881a6f16c365 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:16.289311 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:16.371050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a27dc-a673-4f55-829d-cf71cdd99df9 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:16.376315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0c4a1-b3a3-4428-9f98-881a6f16c365 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:16.401796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:16.415621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d4203-ffcf-4019-a4b4-8cd3ab9521af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:16.415621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d4203-ffcf-4019-a4b4-8cd3ab9521af None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:16.415621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d4203-ffcf-4019-a4b4-8cd3ab9521af None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:16.415621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d4203-ffcf-4019-a4b4-8cd3ab9521af None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:16.416132 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d4203-ffcf-4019-a4b4-8cd3ab9521af None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:16.416132 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d4203-ffcf-4019-a4b4-8cd3ab9521af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:16.416132 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d4203-ffcf-4019-a4b4-8cd3ab9521af None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:16.416326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc1bff-3973-4752-a0b1-ac7ce4428db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:16.416326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc1bff-3973-4752-a0b1-ac7ce4428db6 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:16.416326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc1bff-3973-4752-a0b1-ac7ce4428db6 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:16.417092 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc1bff-3973-4752-a0b1-ac7ce4428db6 None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:16.417918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc1bff-3973-4752-a0b1-ac7ce4428db6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:16.418191 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc1bff-3973-4752-a0b1-ac7ce4428db6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:16.419627 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc1bff-3973-4752-a0b1-ac7ce4428db6 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:16.424806 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 495/979] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 30 14:39:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:16.425528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d4203-ffcf-4019-a4b4-8cd3ab9521af None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:16.442830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 485/980] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Aug 30 14:39:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:16.444061 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc1bff-3973-4752-a0b1-ac7ce4428db6 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:16.533159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:16.537625 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d4203-ffcf-4019-a4b4-8cd3ab9521af None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:16.545811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc1bff-3973-4752-a0b1-ac7ce4428db6 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:16.571441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59234759-70ca-471a-9130-f588db79005a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:16.572039 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59234759-70ca-471a-9130-f588db79005a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:16.572316 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59234759-70ca-471a-9130-f588db79005a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:16.572583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59234759-70ca-471a-9130-f588db79005a None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:16.573284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59234759-70ca-471a-9130-f588db79005a None admin] RBAC: Adding query filter params (interface=admin, service_id=8734061efad0467092aad94c16e9ba96, region_id=RegionOne) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:16.573593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59234759-70ca-471a-9130-f588db79005a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:16.574943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59234759-70ca-471a-9130-f588db79005a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:16.579361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d8e4c-12f6-48a5-838f-b77dcfb1fa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:16.579814 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 496/981] 10.222.0.27 () {56 vars in 1249 bytes} [Tue Aug 30 14:39:16 2022] GET /identity/v3/endpoints?service_id=8734061efad0467092aad94c16e9ba96&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:16.579981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d8e4c-12f6-48a5-838f-b77dcfb1fa2f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:16.580232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d8e4c-12f6-48a5-838f-b77dcfb1fa2f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:16.580500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d8e4c-12f6-48a5-838f-b77dcfb1fa2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:16.580606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59234759-70ca-471a-9130-f588db79005a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:16.639318 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:16.652051 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d8e4c-12f6-48a5-838f-b77dcfb1fa2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:16.653488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d8e4c-12f6-48a5-838f-b77dcfb1fa2f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:16.684071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59234759-70ca-471a-9130-f588db79005a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:16.717186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a9316-626f-4c79-aee1-6a1cbb99b40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:16.717788 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a9316-626f-4c79-aee1-6a1cbb99b40b None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:16.718071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a9316-626f-4c79-aee1-6a1cbb99b40b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:16.718354 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a9316-626f-4c79-aee1-6a1cbb99b40b None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:16.718993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a9316-626f-4c79-aee1-6a1cbb99b40b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:16.720387 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a9316-626f-4c79-aee1-6a1cbb99b40b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:16.738898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:16.750740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 486/982] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:39:16 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 14:39:16.751553 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d8e4c-12f6-48a5-838f-b77dcfb1fa2f None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:16.797645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 497/983] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 30 14:39:16 2022] POST /identity/v3/services => generated 242 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:39:16.798415 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a9316-626f-4c79-aee1-6a1cbb99b40b None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:16.866359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d8e4c-12f6-48a5-838f-b77dcfb1fa2f None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:16.916910 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7c089-5735-4c98-8de1-a17c278ab7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:16.917955 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7c089-5735-4c98-8de1-a17c278ab7ec None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:16.918254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a9316-626f-4c79-aee1-6a1cbb99b40b None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:16.918687 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7c089-5735-4c98-8de1-a17c278ab7ec None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:16.919218 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7c089-5735-4c98-8de1-a17c278ab7ec None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:16.920046 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7c089-5735-4c98-8de1-a17c278ab7ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:16.921551 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7c089-5735-4c98-8de1-a17c278ab7ec None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:16.935038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:16.936283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 487/984] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 30 14:39:16 2022] GET /identity/v3/services => generated 1727 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:39:16.960305 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43481447-fc75-4af3-a314-04176a8df389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:16.960963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43481447-fc75-4af3-a314-04176a8df389 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:16.961242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43481447-fc75-4af3-a314-04176a8df389 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:16.961510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43481447-fc75-4af3-a314-04176a8df389 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:16.975469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43481447-fc75-4af3-a314-04176a8df389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:16.976978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43481447-fc75-4af3-a314-04176a8df389 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:16.993439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-43481447-fc75-4af3-a314-04176a8df389 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:16.993439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:16.993439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:16.993439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:16.993439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:16.993439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:16.993439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:16.993439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:16.993439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:16.993439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:16.993439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:16.993439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:16.993439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:39:16.993439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:39:16.993439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:39:16.993439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:16.994469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:39:16.995520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:39:16.995520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:16.995520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:16.995520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 498/985] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:17.000828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7c089-5735-4c98-8de1-a17c278ab7ec None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:17.056154 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:17.112605 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7c089-5735-4c98-8de1-a17c278ab7ec None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:17.147230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc16055-1386-4b4a-ac08-e77fe6dd4f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:17.147978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc16055-1386-4b4a-ac08-e77fe6dd4f4d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:17.148184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc16055-1386-4b4a-ac08-e77fe6dd4f4d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:17.148423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc16055-1386-4b4a-ac08-e77fe6dd4f4d None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:17.149417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc16055-1386-4b4a-ac08-e77fe6dd4f4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:17.150227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc16055-1386-4b4a-ac08-e77fe6dd4f4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:17.151616 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc16055-1386-4b4a-ac08-e77fe6dd4f4d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:17.163659 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 488/986] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:17.169061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43481447-fc75-4af3-a314-04176a8df389 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:17.277669 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43481447-fc75-4af3-a314-04176a8df389 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:17.312543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408f8b5-781b-4814-95c7-4fe0408d5d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:17.313171 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408f8b5-781b-4814-95c7-4fe0408d5d55 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:17.313447 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408f8b5-781b-4814-95c7-4fe0408d5d55 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:17.313717 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408f8b5-781b-4814-95c7-4fe0408d5d55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:17.320444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408f8b5-781b-4814-95c7-4fe0408d5d55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:17.322409 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408f8b5-781b-4814-95c7-4fe0408d5d55 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:17.333741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-9408f8b5-781b-4814-95c7-4fe0408d5d55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:17.333741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:17.333741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:17.333741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:17.333741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:17.333741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:17.333741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:17.333741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:17.333741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:17.333741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:17.333741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:17.333741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:17.333741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:39:17.333741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:39:17.333741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:17.334813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:17.335918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:39:17.335918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:39:17.335918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:39:17.335918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:39:17.335918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:39:17.335918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:39:17.335918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:17.335918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:17.335918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 499/987] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:39:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:17.341272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc16055-1386-4b4a-ac08-e77fe6dd4f4d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:17.443891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc16055-1386-4b4a-ac08-e77fe6dd4f4d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:17.453795 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:17.478192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e993547-898a-4f63-9519-c055c7789d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:17.478797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e993547-898a-4f63-9519-c055c7789d5b None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:17.479074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e993547-898a-4f63-9519-c055c7789d5b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:17.479369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e993547-898a-4f63-9519-c055c7789d5b None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:17.480058 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e993547-898a-4f63-9519-c055c7789d5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:17.480360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e993547-898a-4f63-9519-c055c7789d5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:17.481789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e993547-898a-4f63-9519-c055c7789d5b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:17.502965 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 489/988] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:39:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:17.504201 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408f8b5-781b-4814-95c7-4fe0408d5d55 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:17.532353 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:17.604441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408f8b5-781b-4814-95c7-4fe0408d5d55 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:17.638764 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf4fdc-a401-4464-895e-1eafa729615b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:17.639460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf4fdc-a401-4464-895e-1eafa729615b None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:17.639800 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf4fdc-a401-4464-895e-1eafa729615b None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:17.640115 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf4fdc-a401-4464-895e-1eafa729615b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:17.640846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf4fdc-a401-4464-895e-1eafa729615b None admin] RBAC: Adding query filter params (role.id=bcecc24215984de9a00652caa5433ffe, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=912e9c732f194fa496a9111bbfab3efa) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:17.641157 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf4fdc-a401-4464-895e-1eafa729615b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:17.643019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf4fdc-a401-4464-895e-1eafa729615b None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:17.650078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 500/989] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:39:17 2022] GET /identity/v3/role_assignments?user.id=912e9c732f194fa496a9111bbfab3efa&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=bcecc24215984de9a00652caa5433ffe => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:18.750969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de45e1-8a1c-4457-b6e1-dddc7c667b25 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:18.751527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de45e1-8a1c-4457-b6e1-dddc7c667b25 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:18.751527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de45e1-8a1c-4457-b6e1-dddc7c667b25 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:18.753012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 490/990] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:18.757881 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ab290-5024-4cc0-b1f8-a966c2fc2d89 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:18.758264 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ab290-5024-4cc0-b1f8-a966c2fc2d89 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:18.758426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ab290-5024-4cc0-b1f8-a966c2fc2d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:18.776375 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a38ab290-5024-4cc0-b1f8-a966c2fc2d89 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:18.777538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a38ab290-5024-4cc0-b1f8-a966c2fc2d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:18.845241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 501/991] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:18.847595 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f44cb6-5083-4bae-941a-b0c887e0cad5 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:18.847945 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f44cb6-5083-4bae-941a-b0c887e0cad5 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:18.848291 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f44cb6-5083-4bae-941a-b0c887e0cad5 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:18.849817 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 491/992] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:18.855462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68458553-9dbe-40e6-8b01-875a001da0aa None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:18.855932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68458553-9dbe-40e6-8b01-875a001da0aa None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:18.856395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68458553-9dbe-40e6-8b01-875a001da0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:18.874809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-68458553-9dbe-40e6-8b01-875a001da0aa None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:18.876254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-68458553-9dbe-40e6-8b01-875a001da0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:18.890729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:18.942504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 502/993] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:19.109162 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f44cb6-5083-4bae-941a-b0c887e0cad5 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:19.208151 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68458553-9dbe-40e6-8b01-875a001da0aa None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:19.212092 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f44cb6-5083-4bae-941a-b0c887e0cad5 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:19.250666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b26436-2ec1-4c63-85a6-2cfc01b43943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:19.251318 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b26436-2ec1-4c63-85a6-2cfc01b43943 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:19.251597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b26436-2ec1-4c63-85a6-2cfc01b43943 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:19.251856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b26436-2ec1-4c63-85a6-2cfc01b43943 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:19.252431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b26436-2ec1-4c63-85a6-2cfc01b43943 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:19.253778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b26436-2ec1-4c63-85a6-2cfc01b43943 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-71b26436-2ec1-4c63-85a6-2cfc01b43943 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:19.263834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:19.264854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:19.264854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:19.264854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:19.264854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:19.264854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:19.264854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:19.264854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:39:19.264854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:39:19.264854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:39:19.264854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:39:19.264854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:19.264854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:19.264854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:19.264854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:19.264854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:19.264854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:19.265715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:19.265715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:19.265715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:19.265715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:19.265715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:19.265715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:19.265715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:19.265715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:19.265715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:19.265715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:19.265715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:39:19.265715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:19.265715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:39:19.265715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:19.266976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 492/994] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:39:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:19.272071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b26436-2ec1-4c63-85a6-2cfc01b43943 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:19.307536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:19.318887 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68458553-9dbe-40e6-8b01-875a001da0aa None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:19.354440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac3eba-f690-433a-91ce-219834b34466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:19.355063 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac3eba-f690-433a-91ce-219834b34466 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:19.355366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac3eba-f690-433a-91ce-219834b34466 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:19.355635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac3eba-f690-433a-91ce-219834b34466 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:19.356226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fac3eba-f690-433a-91ce-219834b34466 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:19.357996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fac3eba-f690-433a-91ce-219834b34466 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-3fac3eba-f690-433a-91ce-219834b34466 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:19.366888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:19.367942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:19.367942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:19.367942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:19.367942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:19.367942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:19.367942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:19.367942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:39:19.367942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:39:19.367942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:39:19.367942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:39:19.367942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:19.367942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:19.367942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:19.367942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:19.367942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:19.367942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:19.369324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:19.369324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:19.369324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:19.369324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:19.369324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:19.369324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:19.369324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:19.369324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:19.369324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:19.369324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:19.369324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:39:19.369324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:19.369324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:39:19.369324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:19.369324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 503/995] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Aug 30 14:39:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:19.374372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac3eba-f690-433a-91ce-219834b34466 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:19.389658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b26436-2ec1-4c63-85a6-2cfc01b43943 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:19.390178 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:19.430919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292ecdc-1dae-4795-9b41-9612945d808b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:19.431799 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292ecdc-1dae-4795-9b41-9612945d808b None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:19.432266 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292ecdc-1dae-4795-9b41-9612945d808b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:19.432541 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292ecdc-1dae-4795-9b41-9612945d808b None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:19.433166 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292ecdc-1dae-4795-9b41-9612945d808b None admin] RBAC: Adding query filter params (name=compute) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:19.433508 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292ecdc-1dae-4795-9b41-9612945d808b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:19.434838 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292ecdc-1dae-4795-9b41-9612945d808b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:19.439991 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 493/996] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 30 14:39:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:19.445415 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292ecdc-1dae-4795-9b41-9612945d808b None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:19.477078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac3eba-f690-433a-91ce-219834b34466 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:19.514005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263f86c-5b4c-4d7a-985e-982e1d80f318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:19.514612 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263f86c-5b4c-4d7a-985e-982e1d80f318 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:19.514891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263f86c-5b4c-4d7a-985e-982e1d80f318 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:19.515190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263f86c-5b4c-4d7a-985e-982e1d80f318 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:19.515826 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263f86c-5b4c-4d7a-985e-982e1d80f318 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:19.516127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263f86c-5b4c-4d7a-985e-982e1d80f318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:19.517496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263f86c-5b4c-4d7a-985e-982e1d80f318 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:19.522618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 504/997] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 30 14:39:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:19.528596 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263f86c-5b4c-4d7a-985e-982e1d80f318 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:19.545764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:19.551611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292ecdc-1dae-4795-9b41-9612945d808b None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:19.588797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c699b-53c1-4a1a-8ac7-729aa579eb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:19.589392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c699b-53c1-4a1a-8ac7-729aa579eb0e None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:19.589665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c699b-53c1-4a1a-8ac7-729aa579eb0e None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:19.589939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c699b-53c1-4a1a-8ac7-729aa579eb0e None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:19.590549 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c699b-53c1-4a1a-8ac7-729aa579eb0e None admin] RBAC: Adding query filter params (type=compute) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:19.590844 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c699b-53c1-4a1a-8ac7-729aa579eb0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:19.592230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c699b-53c1-4a1a-8ac7-729aa579eb0e None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:19.602097 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 494/998] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 30 14:39:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:19.602097 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d2d45-ba88-4158-9b31-588eea26d89f None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:19.602097 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d2d45-ba88-4158-9b31-588eea26d89f None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:19.602097 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d2d45-ba88-4158-9b31-588eea26d89f None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:19.602097 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 495/999] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:19.603911 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d2d45-ba88-4158-9b31-588eea26d89f None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:19.609373 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:19.630559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263f86c-5b4c-4d7a-985e-982e1d80f318 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:19.666180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abec610-7bab-4fca-b409-e0bbadb59b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:19.666779 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abec610-7bab-4fca-b409-e0bbadb59b07 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:19.667058 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abec610-7bab-4fca-b409-e0bbadb59b07 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:19.667361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abec610-7bab-4fca-b409-e0bbadb59b07 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:19.667991 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abec610-7bab-4fca-b409-e0bbadb59b07 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:19.668291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abec610-7bab-4fca-b409-e0bbadb59b07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:19.669699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abec610-7bab-4fca-b409-e0bbadb59b07 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:19.674490 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 505/1000] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Aug 30 14:39:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:19.675907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d963e6-5476-4bad-bd4c-82cdc529ca47 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:19.676185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d963e6-5476-4bad-bd4c-82cdc529ca47 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:19.676452 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d963e6-5476-4bad-bd4c-82cdc529ca47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:19.693662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-54d963e6-5476-4bad-bd4c-82cdc529ca47 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:19.694833 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-54d963e6-5476-4bad-bd4c-82cdc529ca47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:19.704651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d2d45-ba88-4158-9b31-588eea26d89f None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:19.739776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e37b8-57e2-4ae0-98e2-bf48f84b8f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:19.740444 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e37b8-57e2-4ae0-98e2-bf48f84b8f4d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:19.740723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e37b8-57e2-4ae0-98e2-bf48f84b8f4d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:19.740996 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e37b8-57e2-4ae0-98e2-bf48f84b8f4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:19.741694 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e37b8-57e2-4ae0-98e2-bf48f84b8f4d None admin] RBAC: Adding query filter params (interface=public, service_id=636d2aa997f44a82b10dc49d1be6a448, region_id=RegionOne) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:19.742003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e37b8-57e2-4ae0-98e2-bf48f84b8f4d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:19.743907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e37b8-57e2-4ae0-98e2-bf48f84b8f4d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:19.748838 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 496/1001] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Aug 30 14:39:19 2022] GET /identity/v3/endpoints?service_id=636d2aa997f44a82b10dc49d1be6a448&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:19.749582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e37b8-57e2-4ae0-98e2-bf48f84b8f4d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:19.761820 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 506/1002] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:19.762644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d963e6-5476-4bad-bd4c-82cdc529ca47 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:19.776765 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:19.839461 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:19.864310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e37b8-57e2-4ae0-98e2-bf48f84b8f4d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:19.869851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d963e6-5476-4bad-bd4c-82cdc529ca47 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:19.903405 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e09158-a7a0-4a12-883c-824c2ca67ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:19.903935 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e09158-a7a0-4a12-883c-824c2ca67ef0 None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:19.904214 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e09158-a7a0-4a12-883c-824c2ca67ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:19.904484 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e09158-a7a0-4a12-883c-824c2ca67ef0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:19.905154 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e09158-a7a0-4a12-883c-824c2ca67ef0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:19.906488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e09158-a7a0-4a12-883c-824c2ca67ef0 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:19.907049 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005f736-d0a1-4f20-a9f5-8c20c497387a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:19.907679 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005f736-d0a1-4f20-a9f5-8c20c497387a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:19.907962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005f736-d0a1-4f20-a9f5-8c20c497387a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:19.908224 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005f736-d0a1-4f20-a9f5-8c20c497387a None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:19.908832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005f736-d0a1-4f20-a9f5-8c20c497387a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:19.910166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005f736-d0a1-4f20-a9f5-8c20c497387a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:19.916502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 507/1003] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 30 14:39:19 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:39:19.998866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 497/1004] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:39:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:39:20.032509 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005f736-d0a1-4f20-a9f5-8c20c497387a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:20.145412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005f736-d0a1-4f20-a9f5-8c20c497387a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:20.185071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:20.193988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1c07a-d28f-4eef-93ad-8810a3d3f173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:20.193988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1c07a-d28f-4eef-93ad-8810a3d3f173 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:20.193988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1c07a-d28f-4eef-93ad-8810a3d3f173 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:20.193988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1c07a-d28f-4eef-93ad-8810a3d3f173 None admin] PATH_INFO: `/v3/services/image` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:20.198265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1c07a-d28f-4eef-93ad-8810a3d3f173 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:20.198265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1c07a-d28f-4eef-93ad-8810a3d3f173 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-a2f1c07a-d28f-4eef-93ad-8810a3d3f173 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:20.202924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:20.204046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:20.204046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:20.204046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:20.204046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:20.204046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:20.204046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:20.204046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:39:20.204046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:39:20.204046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:39:20.204046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:39:20.204046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:20.204046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:20.204046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:20.204046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:20.204046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:20.204046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:20.204965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:20.204965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:20.204965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:20.204965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:20.204965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:20.204965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:20.204965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:20.204965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:20.204965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:20.204965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:20.204965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:39:20.204965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:20.204965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:39:20.204965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:20.204965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 508/1005] 10.222.0.27 () {56 vars in 1105 bytes} [Tue Aug 30 14:39:20 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:20.212501 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e09158-a7a0-4a12-883c-824c2ca67ef0 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:20.310219 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:20.326805 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e09158-a7a0-4a12-883c-824c2ca67ef0 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:20.400945 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e1195-542f-4e76-ae96-0a20838aa890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:20.401542 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e1195-542f-4e76-ae96-0a20838aa890 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:20.401817 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e1195-542f-4e76-ae96-0a20838aa890 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:20.402084 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e1195-542f-4e76-ae96-0a20838aa890 None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:20.402711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e1195-542f-4e76-ae96-0a20838aa890 None admin] RBAC: Adding query filter params (name=image) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:20.403010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e1195-542f-4e76-ae96-0a20838aa890 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:20.404396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e1195-542f-4e76-ae96-0a20838aa890 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:20.414000 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 498/1006] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:20.416374 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1c07a-d28f-4eef-93ad-8810a3d3f173 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:20.518116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1c07a-d28f-4eef-93ad-8810a3d3f173 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:20.552270 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b38e6-1f06-46e4-840c-06135cff4ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:20.552879 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b38e6-1f06-46e4-840c-06135cff4ba3 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:20.553154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b38e6-1f06-46e4-840c-06135cff4ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:20.553421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b38e6-1f06-46e4-840c-06135cff4ba3 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:20.554040 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b38e6-1f06-46e4-840c-06135cff4ba3 None admin] RBAC: Adding query filter params (type=image) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:20.554337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b38e6-1f06-46e4-840c-06135cff4ba3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:20.555864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b38e6-1f06-46e4-840c-06135cff4ba3 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:20.567200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 509/1007] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:20 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:21.832154 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e89389-65a5-43b0-9333-c65c2a91690d None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:21.832670 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e89389-65a5-43b0-9333-c65c2a91690d None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:21.832781 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e89389-65a5-43b0-9333-c65c2a91690d None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:21.835141 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 499/1008] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:21.839246 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e326168-85e1-4293-a7d9-bf886e4a17ed None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:21.839647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e326168-85e1-4293-a7d9-bf886e4a17ed None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:21.839827 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e326168-85e1-4293-a7d9-bf886e4a17ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:21.858090 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3e326168-85e1-4293-a7d9-bf886e4a17ed None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:21.859285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3e326168-85e1-4293-a7d9-bf886e4a17ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:21.915630 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:21.926568 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 510/1009] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:21 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:21.937392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0572ba-29a2-46e1-9036-204fa3ab2c1d None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:21.937783 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0572ba-29a2-46e1-9036-204fa3ab2c1d None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:21.938065 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0572ba-29a2-46e1-9036-204fa3ab2c1d None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:21.939553 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 500/1010] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:21.944393 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df247fce-813d-4740-90b2-ca48cb5b7e9e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:21.944688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df247fce-813d-4740-90b2-ca48cb5b7e9e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:21.944965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df247fce-813d-4740-90b2-ca48cb5b7e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:21.964385 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-df247fce-813d-4740-90b2-ca48cb5b7e9e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:21.965574 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-df247fce-813d-4740-90b2-ca48cb5b7e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:22.034436 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 511/1011] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:21 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:22.196988 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0572ba-29a2-46e1-9036-204fa3ab2c1d None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:22.307079 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0572ba-29a2-46e1-9036-204fa3ab2c1d None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:22.308785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df247fce-813d-4740-90b2-ca48cb5b7e9e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:22.382482 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50adae66-8f07-4e0f-8ff5-30c446468991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:22.383134 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adae66-8f07-4e0f-8ff5-30c446468991 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:22.383932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adae66-8f07-4e0f-8ff5-30c446468991 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:22.384208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adae66-8f07-4e0f-8ff5-30c446468991 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:22.384777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50adae66-8f07-4e0f-8ff5-30c446468991 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:22.386150 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50adae66-8f07-4e0f-8ff5-30c446468991 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-50adae66-8f07-4e0f-8ff5-30c446468991 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:22.395059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:22.396072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:22.396072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:22.396072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:22.396072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:22.396072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:22.396072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:22.396072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:39:22.396072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:39:22.396072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:39:22.396072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:39:22.396072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:22.396072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:22.396072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:22.396072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:22.396072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:22.396072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:22.396918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:22.396918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:22.396918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:22.396918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:22.396918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:22.396918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:22.396918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:22.396918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:22.396918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:22.396918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:22.396918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:39:22.396918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:22.396918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:39:22.396918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:22.396918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 501/1012] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:39:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:22.402673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50adae66-8f07-4e0f-8ff5-30c446468991 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:22.417398 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:22.421959 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df247fce-813d-4740-90b2-ca48cb5b7e9e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:22.444298 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:22.460744 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8765fb-5133-436f-9467-419ad5c68648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:22.461311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8765fb-5133-436f-9467-419ad5c68648 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:22.462140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8765fb-5133-436f-9467-419ad5c68648 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:22.462453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8765fb-5133-436f-9467-419ad5c68648 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:22.463769 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8765fb-5133-436f-9467-419ad5c68648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:22.465205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8765fb-5133-436f-9467-419ad5c68648 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:22.467142 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 512/1013] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:22.467972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71ae20-90b6-4947-96ba-52a5d096f978 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:22.468254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71ae20-90b6-4947-96ba-52a5d096f978 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:22.468525 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71ae20-90b6-4947-96ba-52a5d096f978 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:22.469976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 513/1014] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:22.472406 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71ae20-90b6-4947-96ba-52a5d096f978 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:22.502957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50adae66-8f07-4e0f-8ff5-30c446468991 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:22.537515 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d047b7-275f-46b0-8667-3ac3fb07c1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:22.538122 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d047b7-275f-46b0-8667-3ac3fb07c1b4 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:22.538401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d047b7-275f-46b0-8667-3ac3fb07c1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:22.538691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d047b7-275f-46b0-8667-3ac3fb07c1b4 None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:22.539335 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d047b7-275f-46b0-8667-3ac3fb07c1b4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:22.539661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d047b7-275f-46b0-8667-3ac3fb07c1b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:22.541025 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d047b7-275f-46b0-8667-3ac3fb07c1b4 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:22.545842 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 502/1015] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 30 14:39:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:22.546787 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ec416-da42-4972-b750-035cde0c5049 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:22.547070 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ec416-da42-4972-b750-035cde0c5049 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:22.547369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ec416-da42-4972-b750-035cde0c5049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:22.564658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-eb2ec416-da42-4972-b750-035cde0c5049 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:22.565842 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-eb2ec416-da42-4972-b750-035cde0c5049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:22.576076 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71ae20-90b6-4947-96ba-52a5d096f978 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:22.609452 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a393a-bbc1-455f-b39b-1e83610fed0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:22.610129 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a393a-bbc1-455f-b39b-1e83610fed0d None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:22.610412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a393a-bbc1-455f-b39b-1e83610fed0d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:22.610678 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a393a-bbc1-455f-b39b-1e83610fed0d None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:22.611339 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a393a-bbc1-455f-b39b-1e83610fed0d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:22.612747 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a393a-bbc1-455f-b39b-1e83610fed0d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:22.631949 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 503/1016] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:22 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:22.632770 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ec416-da42-4972-b750-035cde0c5049 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:22.644972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:22.727252 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 514/1017] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 30 14:39:22 2022] POST /identity/v3/projects => generated 311 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:39:22.734633 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ec416-da42-4972-b750-035cde0c5049 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:22.774959 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb46369-b7c5-4e21-8487-1c4c430fa2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:22.775619 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb46369-b7c5-4e21-8487-1c4c430fa2e8 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:22.775901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb46369-b7c5-4e21-8487-1c4c430fa2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:22.776170 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb46369-b7c5-4e21-8487-1c4c430fa2e8 None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:22.776886 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb46369-b7c5-4e21-8487-1c4c430fa2e8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:22.777185 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb46369-b7c5-4e21-8487-1c4c430fa2e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:22.778562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb46369-b7c5-4e21-8487-1c4c430fa2e8 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:22.784373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 504/1018] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Aug 30 14:39:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:22.790159 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a393a-bbc1-455f-b39b-1e83610fed0d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:22.835324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:22.898802 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a393a-bbc1-455f-b39b-1e83610fed0d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:22.901336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb46369-b7c5-4e21-8487-1c4c430fa2e8 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:22.937036 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6494847-484d-4d39-bbd8-ac31aecf2492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:22.937974 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6494847-484d-4d39-bbd8-ac31aecf2492 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:22.938447 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6494847-484d-4d39-bbd8-ac31aecf2492 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:22.938907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6494847-484d-4d39-bbd8-ac31aecf2492 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:22.939792 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6494847-484d-4d39-bbd8-ac31aecf2492 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:22.941348 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6494847-484d-4d39-bbd8-ac31aecf2492 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:22.976683 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:23.007757 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb46369-b7c5-4e21-8487-1c4c430fa2e8 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:23.035131 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 515/1019] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:39:22 2022] POST /identity/v3/endpoints => generated 334 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:39:23.059845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d187f-be9d-47f1-84dd-92c5cd66b46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:23.060460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d187f-be9d-47f1-84dd-92c5cd66b46b None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:23.060733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d187f-be9d-47f1-84dd-92c5cd66b46b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:23.060996 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d187f-be9d-47f1-84dd-92c5cd66b46b None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:23.061650 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d187f-be9d-47f1-84dd-92c5cd66b46b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:23.063042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d187f-be9d-47f1-84dd-92c5cd66b46b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:23.146578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 505/1020] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 30 14:39:22 2022] POST /identity/v3/services => generated 242 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:39:24.681814 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fda0d2-dccc-4461-9102-52c2581d0ed4 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:24.682296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fda0d2-dccc-4461-9102-52c2581d0ed4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:24.682296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fda0d2-dccc-4461-9102-52c2581d0ed4 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:24.683714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 516/1021] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:24.688850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86fceb-9d82-4bc1-b97d-f53deb3787af None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:24.689270 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86fceb-9d82-4bc1-b97d-f53deb3787af None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:24.689438 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86fceb-9d82-4bc1-b97d-f53deb3787af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:24.707768 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-2c86fceb-9d82-4bc1-b97d-f53deb3787af None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:24.708951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-2c86fceb-9d82-4bc1-b97d-f53deb3787af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:24.784286 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 506/1022] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:25.044053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c3081-dece-47a6-82c0-5ec209ef7699 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:25.044393 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c3081-dece-47a6-82c0-5ec209ef7699 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:25.044606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c3081-dece-47a6-82c0-5ec209ef7699 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:25.045986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 517/1023] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:25.051041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d2f35-14d7-40b1-a384-4ca628ed58ec None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:25.051353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d2f35-14d7-40b1-a384-4ca628ed58ec None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:25.051630 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d2f35-14d7-40b1-a384-4ca628ed58ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:25.057290 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c3081-dece-47a6-82c0-5ec209ef7699 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:25.070031 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-669d2f35-14d7-40b1-a384-4ca628ed58ec None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:25.071300 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-669d2f35-14d7-40b1-a384-4ca628ed58ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:25.118193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:25.137093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 507/1024] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:25.149756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc0853-41f7-4af4-92cc-d723fb5b5568 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:25.149756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc0853-41f7-4af4-92cc-d723fb5b5568 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:25.149756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc0853-41f7-4af4-92cc-d723fb5b5568 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:25.149756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 508/1025] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:25.149756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97edbfe-32be-4fc2-b774-57f5f364fc1a None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:25.149756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97edbfe-32be-4fc2-b774-57f5f364fc1a None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:25.150264 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97edbfe-32be-4fc2-b774-57f5f364fc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:25.159761 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:25.166509 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-c97edbfe-32be-4fc2-b774-57f5f364fc1a None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:25.167407 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c3081-dece-47a6-82c0-5ec209ef7699 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:25.167719 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c97edbfe-32be-4fc2-b774-57f5f364fc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:25.201872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdefb96-8dcd-4772-a8f2-f1c9e379642a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:25.202912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdefb96-8dcd-4772-a8f2-f1c9e379642a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:25.204293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdefb96-8dcd-4772-a8f2-f1c9e379642a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:25.204725 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdefb96-8dcd-4772-a8f2-f1c9e379642a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:25.205850 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdefb96-8dcd-4772-a8f2-f1c9e379642a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:25.207322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdefb96-8dcd-4772-a8f2-f1c9e379642a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:25.209051 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 518/1026] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:25.215172 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdefb96-8dcd-4772-a8f2-f1c9e379642a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:25.237889 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 509/1027] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:39:25.323145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdefb96-8dcd-4772-a8f2-f1c9e379642a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:25.356682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02615e7c-974f-420a-b07b-781b28a17023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:25.357283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02615e7c-974f-420a-b07b-781b28a17023 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:25.357567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02615e7c-974f-420a-b07b-781b28a17023 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:25.361421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02615e7c-974f-420a-b07b-781b28a17023 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:25.361421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02615e7c-974f-420a-b07b-781b28a17023 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:25.361421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02615e7c-974f-420a-b07b-781b28a17023 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:25.408703 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97edbfe-32be-4fc2-b774-57f5f364fc1a None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:25.463178 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 519/1028] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:39:25 2022] POST /identity/v3/users => generated 285 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:39:25.507357 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02615e7c-974f-420a-b07b-781b28a17023 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:25.514654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97edbfe-32be-4fc2-b774-57f5f364fc1a None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:25.529940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:25.536695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:25.555221 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8298269-c0f1-44bd-8429-0694b932480f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:25.555875 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8298269-c0f1-44bd-8429-0694b932480f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:25.556154 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8298269-c0f1-44bd-8429-0694b932480f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:25.556425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8298269-c0f1-44bd-8429-0694b932480f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:25.568153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8298269-c0f1-44bd-8429-0694b932480f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:39:25.579239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-f8298269-c0f1-44bd-8429-0694b932480f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:39:25.579239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:25.579239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:25.579239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:25.579239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:25.579239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:25.579239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:25.579239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:25.579239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:25.579239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:25.579239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:25.579239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:25.579239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:39:25.579239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:39:25.579239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:39:25.579239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:39:25.580253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:39:25.580253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:39:25.580253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:39:25.580253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:39:25.580253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:25.580253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:25.580253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:25.580253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:25.580253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:25.580253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:25.580253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:25.580253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:25.580253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:25.580253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:25.580253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:25.580253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:25.581131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:25.581131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:25.581131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:25.581131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:25.581131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:39:25.581131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:39:25.581131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:39:25.581131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:39:25.581131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:39:25.581131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:39:25.581131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:39:25.581131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:25.581131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 510/1029] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Aug 30 14:39:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:25.582189 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8298269-c0f1-44bd-8429-0694b932480f None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:25.628723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02615e7c-974f-420a-b07b-781b28a17023 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:25.668405 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9429cf8-4e39-45ca-9937-0720a4b3cbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:25.669044 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9429cf8-4e39-45ca-9937-0720a4b3cbe4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:25.669321 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9429cf8-4e39-45ca-9937-0720a4b3cbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:25.669594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9429cf8-4e39-45ca-9937-0720a4b3cbe4 None admin] PATH_INFO: `/v3/services/image` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:25.670188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9429cf8-4e39-45ca-9937-0720a4b3cbe4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:25.671626 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9429cf8-4e39-45ca-9937-0720a4b3cbe4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-f9429cf8-4e39-45ca-9937-0720a4b3cbe4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:25.681126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:25.682581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:25.682581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:25.682581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:25.682581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:25.682581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:25.682581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:25.682581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:39:25.682581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:39:25.682581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:39:25.682581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:39:25.682581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:25.682581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:25.682581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:25.682581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:25.682581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:25.682581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:25.683512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:25.683512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:25.683512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:25.683512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:25.683512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:25.683512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:25.683512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:25.683512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:25.683512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:25.683512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:25.683512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:39:25.683512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:25.683512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:39:25.683512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:25.683512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 520/1030] 10.222.0.27 () {56 vars in 1105 bytes} [Tue Aug 30 14:39:25 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:25.688196 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9429cf8-4e39-45ca-9937-0720a4b3cbe4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:25.696961 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8298269-c0f1-44bd-8429-0694b932480f None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:25.734092 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d26f53-dbc2-49e8-bf44-84ff6f55d920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:25.734691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d26f53-dbc2-49e8-bf44-84ff6f55d920 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:25.734978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d26f53-dbc2-49e8-bf44-84ff6f55d920 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:25.735265 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d26f53-dbc2-49e8-bf44-84ff6f55d920 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:25.735924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d26f53-dbc2-49e8-bf44-84ff6f55d920 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:25.736225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d26f53-dbc2-49e8-bf44-84ff6f55d920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:25.737582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d26f53-dbc2-49e8-bf44-84ff6f55d920 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:25.746425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 511/1031] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Aug 30 14:39:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:25.752649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d26f53-dbc2-49e8-bf44-84ff6f55d920 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:25.776446 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:25.789233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:25.793498 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9429cf8-4e39-45ca-9937-0720a4b3cbe4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:25.829583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88421c79-f8ee-4b11-bb27-9fe1b44f04bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:25.830102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88421c79-f8ee-4b11-bb27-9fe1b44f04bb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:25.830375 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88421c79-f8ee-4b11-bb27-9fe1b44f04bb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:25.830641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88421c79-f8ee-4b11-bb27-9fe1b44f04bb None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:25.831301 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88421c79-f8ee-4b11-bb27-9fe1b44f04bb None admin] RBAC: Adding query filter params (name=image) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:25.831600 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88421c79-f8ee-4b11-bb27-9fe1b44f04bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:25.833027 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88421c79-f8ee-4b11-bb27-9fe1b44f04bb None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:25.842935 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 521/1032] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:25.843616 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88421c79-f8ee-4b11-bb27-9fe1b44f04bb None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:25.857741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d26f53-dbc2-49e8-bf44-84ff6f55d920 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:25.893879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e4840-0cca-482c-8588-4d0b4802d174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:25.894582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e4840-0cca-482c-8588-4d0b4802d174 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:25.894924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e4840-0cca-482c-8588-4d0b4802d174 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:25.895291 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e4840-0cca-482c-8588-4d0b4802d174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:25.906068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e4840-0cca-482c-8588-4d0b4802d174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:25.907755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e4840-0cca-482c-8588-4d0b4802d174 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:25.922230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-786e4840-0cca-482c-8588-4d0b4802d174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:25.922230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:25.922230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:25.922230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:25.922230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:25.922230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:25.922230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:25.922230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:25.922230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:25.922230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:25.922230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:25.922230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:25.922230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:39:25.922230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:39:25.922230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:39:25.922230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:25.923436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:39:25.924684 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:39:25.924684 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:25.924684 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:25.924684 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 512/1033] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:25.929310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e4840-0cca-482c-8588-4d0b4802d174 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:25.947634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88421c79-f8ee-4b11-bb27-9fe1b44f04bb None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:25.984572 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56604d14-265c-4799-8025-2e39758318dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:25.985286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56604d14-265c-4799-8025-2e39758318dc None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:25.985560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56604d14-265c-4799-8025-2e39758318dc None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:25.985943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56604d14-265c-4799-8025-2e39758318dc None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:25.986569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56604d14-265c-4799-8025-2e39758318dc None admin] RBAC: Adding query filter params (type=image) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:25.986868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56604d14-265c-4799-8025-2e39758318dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:25.988252 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56604d14-265c-4799-8025-2e39758318dc None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:25.994047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 522/1034] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:25.999577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56604d14-265c-4799-8025-2e39758318dc None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:26.015274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:26.030370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:26.047040 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e4840-0cca-482c-8588-4d0b4802d174 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:26.087372 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5af1f-7b0b-47d7-94f9-cb183834b9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:26.088055 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5af1f-7b0b-47d7-94f9-cb183834b9cb None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:26.088175 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5af1f-7b0b-47d7-94f9-cb183834b9cb None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:26.088429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5af1f-7b0b-47d7-94f9-cb183834b9cb None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:26.089080 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5af1f-7b0b-47d7-94f9-cb183834b9cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:26.089374 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5af1f-7b0b-47d7-94f9-cb183834b9cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:26.090735 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5af1f-7b0b-47d7-94f9-cb183834b9cb None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:26.103338 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 513/1035] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:26.108590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5af1f-7b0b-47d7-94f9-cb183834b9cb None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:26.110949 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56604d14-265c-4799-8025-2e39758318dc None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:26.149760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dfae2b-d99f-4468-976f-a5b5f3db2622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:26.150365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfae2b-d99f-4468-976f-a5b5f3db2622 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:26.150641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfae2b-d99f-4468-976f-a5b5f3db2622 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:26.150911 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfae2b-d99f-4468-976f-a5b5f3db2622 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:26.151641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dfae2b-d99f-4468-976f-a5b5f3db2622 None admin] RBAC: Adding query filter params (interface=public, service_id=b4fa9294d6ac43009e07c039be869c91, region_id=RegionOne) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:26.151945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dfae2b-d99f-4468-976f-a5b5f3db2622 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:26.153305 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dfae2b-d99f-4468-976f-a5b5f3db2622 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:26.160026 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 523/1036] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Aug 30 14:39:25 2022] GET /identity/v3/endpoints?service_id=b4fa9294d6ac43009e07c039be869c91&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:26.166124 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dfae2b-d99f-4468-976f-a5b5f3db2622 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:26.215119 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5af1f-7b0b-47d7-94f9-cb183834b9cb None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:26.248324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:26.256917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a09d978-7f56-4ce1-9df9-779528595612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:26.257615 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09d978-7f56-4ce1-9df9-779528595612 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:26.257944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09d978-7f56-4ce1-9df9-779528595612 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:26.258801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09d978-7f56-4ce1-9df9-779528595612 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:26.272096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a09d978-7f56-4ce1-9df9-779528595612 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:26.272591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dfae2b-d99f-4468-976f-a5b5f3db2622 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:26.274515 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a09d978-7f56-4ce1-9df9-779528595612 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:26.285920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-3a09d978-7f56-4ce1-9df9-779528595612 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:39:26.285920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:26.285920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:26.285920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:26.285920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:26.285920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:26.285920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:26.285920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:26.285920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:26.285920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:26.285920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:26.285920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:26.285920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:39:26.285920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:39:26.285920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:39:26.285920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:26.287086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:26.288459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:26.288459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:26.288459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:39:26.288459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:39:26.288459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:39:26.288459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:39:26.288459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:39:26.288459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:39:26.288459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:39:26.288459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:26.293074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 514/1037] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 30 14:39:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:26.293465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a09d978-7f56-4ce1-9df9-779528595612 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:26.303626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:26.310369 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888536ed-32c2-46ee-8cea-879e8a8c7ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:26.311070 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888536ed-32c2-46ee-8cea-879e8a8c7ced None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:26.312611 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888536ed-32c2-46ee-8cea-879e8a8c7ced None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:26.312611 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888536ed-32c2-46ee-8cea-879e8a8c7ced None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:26.312611 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888536ed-32c2-46ee-8cea-879e8a8c7ced None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:26.313620 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888536ed-32c2-46ee-8cea-879e8a8c7ced None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:26.320553 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 524/1038] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 30 14:39:26 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:39:26.408325 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a09d978-7f56-4ce1-9df9-779528595612 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:26.444579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5837440-8253-45bd-a979-37fbd3ae402c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:26.445198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5837440-8253-45bd-a979-37fbd3ae402c None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:26.445479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5837440-8253-45bd-a979-37fbd3ae402c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:26.446342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5837440-8253-45bd-a979-37fbd3ae402c None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:26.447093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5837440-8253-45bd-a979-37fbd3ae402c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:26.447427 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5837440-8253-45bd-a979-37fbd3ae402c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:26.448876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5837440-8253-45bd-a979-37fbd3ae402c None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:26.471075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 515/1039] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 30 14:39:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:26.476945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888536ed-32c2-46ee-8cea-879e8a8c7ced None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:26.584775 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888536ed-32c2-46ee-8cea-879e8a8c7ced None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:26.620988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ff687-90be-48d0-901e-776ab550716f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:26.621590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ff687-90be-48d0-901e-776ab550716f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:26.621863 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ff687-90be-48d0-901e-776ab550716f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:26.622128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ff687-90be-48d0-901e-776ab550716f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:26.632603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:26.690627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ff687-90be-48d0-901e-776ab550716f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:26.692022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ff687-90be-48d0-901e-776ab550716f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:26.796139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 525/1040] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:39:26 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:39:26.797440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5837440-8253-45bd-a979-37fbd3ae402c None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:26.862957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:26.902449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5837440-8253-45bd-a979-37fbd3ae402c None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:26.937130 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9f721-4538-4505-b781-cca11e9be93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:26.937761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9f721-4538-4505-b781-cca11e9be93e None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:26.938039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9f721-4538-4505-b781-cca11e9be93e None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:26.938310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9f721-4538-4505-b781-cca11e9be93e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:26.949169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9f721-4538-4505-b781-cca11e9be93e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:26.950545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9f721-4538-4505-b781-cca11e9be93e None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:26.964753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-32d9f721-4538-4505-b781-cca11e9be93e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:26.964753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:26.964753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:26.964753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:26.964753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:26.964753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:26.964753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:26.964753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:26.964753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:26.964753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:26.964753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:26.964753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:26.964753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:39:26.964753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:39:26.964753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:39:26.964753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:26.965739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:39:26.966644 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:39:26.966644 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:26.966644 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:26.966644 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 516/1041] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:26.971652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ff687-90be-48d0-901e-776ab550716f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:27.013302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:27.071336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ff687-90be-48d0-901e-776ab550716f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:27.105921 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84429d13-a5c6-42fc-8e61-3b66f7d17852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:27.106670 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84429d13-a5c6-42fc-8e61-3b66f7d17852 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:27.107054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84429d13-a5c6-42fc-8e61-3b66f7d17852 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:27.107453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84429d13-a5c6-42fc-8e61-3b66f7d17852 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:27.108203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84429d13-a5c6-42fc-8e61-3b66f7d17852 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:27.108610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84429d13-a5c6-42fc-8e61-3b66f7d17852 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:27.110080 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84429d13-a5c6-42fc-8e61-3b66f7d17852 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:27.126706 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 526/1042] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:27.127939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9f721-4538-4505-b781-cca11e9be93e None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:27.238950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9f721-4538-4505-b781-cca11e9be93e None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:27.256239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:27.279322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976aab8d-d152-4866-a58a-8953d631688a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:27.279322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976aab8d-d152-4866-a58a-8953d631688a None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:27.279552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976aab8d-d152-4866-a58a-8953d631688a None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:27.279778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976aab8d-d152-4866-a58a-8953d631688a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:27.287074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976aab8d-d152-4866-a58a-8953d631688a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:27.288544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976aab8d-d152-4866-a58a-8953d631688a None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:27.300607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-976aab8d-d152-4866-a58a-8953d631688a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:27.300607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:27.300607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:27.300607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:27.300607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:27.300607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:27.300607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:27.300607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:27.300607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:27.300607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:27.300607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:27.300607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:27.300607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:39:27.300607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:39:27.300607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:27.302796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:27.303950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:39:27.303950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:39:27.303950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:39:27.303950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:39:27.303950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:39:27.303950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:39:27.303950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:27.303950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:27.303950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 517/1043] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:39:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:27.310730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84429d13-a5c6-42fc-8e61-3b66f7d17852 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:27.412342 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:27.416412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84429d13-a5c6-42fc-8e61-3b66f7d17852 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:27.425929 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0ca4f-7dcf-4080-8cd7-6299bd7b5792 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:27.429629 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0ca4f-7dcf-4080-8cd7-6299bd7b5792 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:27.429629 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0ca4f-7dcf-4080-8cd7-6299bd7b5792 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:27.429629 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 518/1044] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:39:27.432680 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b8f7b-4ef2-4784-99bd-8550678ec24f None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:27.432993 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b8f7b-4ef2-4784-99bd-8550678ec24f None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:27.433273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b8f7b-4ef2-4784-99bd-8550678ec24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:27.450504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1453b-b5f8-495a-b48b-5d690cd436af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:27.451138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1453b-b5f8-495a-b48b-5d690cd436af None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:27.451414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1453b-b5f8-495a-b48b-5d690cd436af None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:27.451688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1453b-b5f8-495a-b48b-5d690cd436af None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:27.452381 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1453b-b5f8-495a-b48b-5d690cd436af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:27.452597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-912b8f7b-4ef2-4784-99bd-8550678ec24f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:27.452703 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1453b-b5f8-495a-b48b-5d690cd436af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:27.453870 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-912b8f7b-4ef2-4784-99bd-8550678ec24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:27.454106 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1453b-b5f8-495a-b48b-5d690cd436af None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:27.472372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 527/1045] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:39:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:27.473407 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1453b-b5f8-495a-b48b-5d690cd436af None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:27.531516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 519/1046] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:39:27.596478 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1453b-b5f8-495a-b48b-5d690cd436af None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:27.628493 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5dfd44-8901-4d0f-af09-0a7f5deb91a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:27.629092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5dfd44-8901-4d0f-af09-0a7f5deb91a0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:27.629369 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5dfd44-8901-4d0f-af09-0a7f5deb91a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:27.629639 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5dfd44-8901-4d0f-af09-0a7f5deb91a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:27.630356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5dfd44-8901-4d0f-af09-0a7f5deb91a0 None admin] RBAC: Adding query filter params (role.id=bcecc24215984de9a00652caa5433ffe, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=5b6e4c5bf2a54b28a55e5ae74b90ab4f) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:27.630661 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5dfd44-8901-4d0f-af09-0a7f5deb91a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:27.632104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5dfd44-8901-4d0f-af09-0a7f5deb91a0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:27.643367 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 528/1047] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:39:27 2022] GET /identity/v3/role_assignments?user.id=5b6e4c5bf2a54b28a55e5ae74b90ab4f&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=bcecc24215984de9a00652caa5433ffe => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:27.795667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b8f7b-4ef2-4784-99bd-8550678ec24f None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:27.894842 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b8f7b-4ef2-4784-99bd-8550678ec24f None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:27.904166 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:27.936470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50dab9-ea55-4c7d-b653-03ed0a3c0112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:27.937104 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50dab9-ea55-4c7d-b653-03ed0a3c0112 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:27.937386 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50dab9-ea55-4c7d-b653-03ed0a3c0112 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:27.937655 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50dab9-ea55-4c7d-b653-03ed0a3c0112 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:27.948329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50dab9-ea55-4c7d-b653-03ed0a3c0112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:39:27.953089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-aa50dab9-ea55-4c7d-b653-03ed0a3c0112 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:39:27.953089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:27.953089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:27.953089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:27.953089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:27.953089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:27.953089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:27.953089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:27.953089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:27.953089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:27.953089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:27.953089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:27.953089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:39:27.953089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:39:27.953089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:39:27.953089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:39:27.953989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:39:27.953989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:39:27.953989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:39:27.953989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:39:27.953989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:27.953989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:27.953989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:27.953989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:27.953989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:27.953989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:27.953989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:27.953989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:27.953989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:27.953989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:27.953989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:27.953989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:27.954856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:27.954856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:27.954856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:27.954856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:27.954856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:39:27.954856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:39:27.954856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:39:27.954856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:39:27.954856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:39:27.954856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:39:27.954856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:39:27.954856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:27.956171 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 520/1048] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:39:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:27.960186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5dfd44-8901-4d0f-af09-0a7f5deb91a0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:27.987201 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:28.065399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5dfd44-8901-4d0f-af09-0a7f5deb91a0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:28.100604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99927d-8431-4667-a0ce-0b0f7f5f7a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:28.101272 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99927d-8431-4667-a0ce-0b0f7f5f7a4c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:28.101551 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99927d-8431-4667-a0ce-0b0f7f5f7a4c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:28.102438 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99927d-8431-4667-a0ce-0b0f7f5f7a4c None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:28.103265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99927d-8431-4667-a0ce-0b0f7f5f7a4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:28.103579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99927d-8431-4667-a0ce-0b0f7f5f7a4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:28.104967 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99927d-8431-4667-a0ce-0b0f7f5f7a4c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:28.115486 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 529/1049] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:39:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:28.121174 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50dab9-ea55-4c7d-b653-03ed0a3c0112 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:28.224707 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50dab9-ea55-4c7d-b653-03ed0a3c0112 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:28.238030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fab15b-8d43-4c74-a14e-18438ba3b8ab None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:28.238327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fab15b-8d43-4c74-a14e-18438ba3b8ab None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:28.238590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fab15b-8d43-4c74-a14e-18438ba3b8ab None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:28.239988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 530/1050] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:39:28.245287 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a72978-4f73-4ca0-a5af-62e3ac5b02e3 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:28.245561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a72978-4f73-4ca0-a5af-62e3ac5b02e3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:28.245831 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a72978-4f73-4ca0-a5af-62e3ac5b02e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:28.264399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a2a72978-4f73-4ca0-a5af-62e3ac5b02e3 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:28.265694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a2a72978-4f73-4ca0-a5af-62e3ac5b02e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:28.277873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ccdb5-39d2-4fe4-bbca-85204813938d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:28.278694 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ccdb5-39d2-4fe4-bbca-85204813938d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:28.279153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ccdb5-39d2-4fe4-bbca-85204813938d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:28.279574 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ccdb5-39d2-4fe4-bbca-85204813938d None admin] PATH_INFO: `/v3/users/5d06a92fb2a5461186204f37e327f5b8` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:28.294467 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ccdb5-39d2-4fe4-bbca-85204813938d None admin] RBAC: Authorizing `identity:get_user(user_id=5d06a92fb2a5461186204f37e327f5b8)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:28.296151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ccdb5-39d2-4fe4-bbca-85204813938d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:28.297938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 521/1051] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:39:28 2022] GET /identity/v3/users/5d06a92fb2a5461186204f37e327f5b8 => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:28.303975 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ccdb5-39d2-4fe4-bbca-85204813938d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:28.330915 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:28.340421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 531/1052] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:39:28.407790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ccdb5-39d2-4fe4-bbca-85204813938d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:28.441716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152d8e7-6d1a-455e-a881-455a56bf4294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:28.442536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152d8e7-6d1a-455e-a881-455a56bf4294 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:28.442999 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152d8e7-6d1a-455e-a881-455a56bf4294 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:28.443484 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152d8e7-6d1a-455e-a881-455a56bf4294 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:28.512084 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152d8e7-6d1a-455e-a881-455a56bf4294 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:28.513792 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152d8e7-6d1a-455e-a881-455a56bf4294 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:28.538331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:28.607407 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a72978-4f73-4ca0-a5af-62e3ac5b02e3 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:28.621388 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 522/1053] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:39:28 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:39:28.627038 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152d8e7-6d1a-455e-a881-455a56bf4294 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:28.629977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:28.717240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a72978-4f73-4ca0-a5af-62e3ac5b02e3 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:28.746076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152d8e7-6d1a-455e-a881-455a56bf4294 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:28.752884 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e32f3b-a43d-4705-a5bd-8b8f03a52fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:28.753910 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e32f3b-a43d-4705-a5bd-8b8f03a52fcf None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:28.754186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e32f3b-a43d-4705-a5bd-8b8f03a52fcf None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:28.754453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e32f3b-a43d-4705-a5bd-8b8f03a52fcf None admin] PATH_INFO: `/v3/services/image` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:28.755408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e32f3b-a43d-4705-a5bd-8b8f03a52fcf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:28.757140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e32f3b-a43d-4705-a5bd-8b8f03a52fcf None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-94e32f3b-a43d-4705-a5bd-8b8f03a52fcf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:28.766463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:28.767675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:28.767675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:28.767675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:28.767675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:28.767675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:28.767675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:28.767675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:39:28.767675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:39:28.767675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:39:28.767675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:39:28.767675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:28.767675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:28.767675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:28.767675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:28.767675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:28.767675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:28.768541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:28.768541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:28.768541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:28.768541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:28.768541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:28.768541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:28.768541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:28.768541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:28.768541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:28.768541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:28.768541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:39:28.768541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:39:28.768541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:39:28.768541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:28.768541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 532/1054] 10.222.0.27 () {56 vars in 1105 bytes} [Tue Aug 30 14:39:28 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:28.773834 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e32f3b-a43d-4705-a5bd-8b8f03a52fcf None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:28.775405 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:28.785958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0bc20-626c-4c03-8a3a-611398cba940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:28.786623 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0bc20-626c-4c03-8a3a-611398cba940 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:28.787068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0bc20-626c-4c03-8a3a-611398cba940 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:28.787195 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0bc20-626c-4c03-8a3a-611398cba940 None admin] PATH_INFO: `/v3/projects/4423f9c79d5048afbd9b0c6eae0bdde1` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:28.788532 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0bc20-626c-4c03-8a3a-611398cba940 None admin] RBAC: Authorizing `identity:get_project(project_id=4423f9c79d5048afbd9b0c6eae0bdde1)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:28.790383 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0bc20-626c-4c03-8a3a-611398cba940 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:28.791965 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 523/1055] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:39:28 2022] GET /identity/v3/projects/4423f9c79d5048afbd9b0c6eae0bdde1 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:28.797774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0bc20-626c-4c03-8a3a-611398cba940 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:28.857292 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:28.876361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e32f3b-a43d-4705-a5bd-8b8f03a52fcf None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:28.907155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0bc20-626c-4c03-8a3a-611398cba940 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:28.912116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc20b6-8dab-4ac3-aeb5-45130fed1c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:28.912725 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc20b6-8dab-4ac3-aeb5-45130fed1c06 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:28.913004 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc20b6-8dab-4ac3-aeb5-45130fed1c06 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:28.913278 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc20b6-8dab-4ac3-aeb5-45130fed1c06 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:28.913921 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc20b6-8dab-4ac3-aeb5-45130fed1c06 None admin] RBAC: Adding query filter params (name=image) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:28.914557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc20b6-8dab-4ac3-aeb5-45130fed1c06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:28.915847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc20b6-8dab-4ac3-aeb5-45130fed1c06 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:28.921050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 533/1056] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:28.926541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc20b6-8dab-4ac3-aeb5-45130fed1c06 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:28.942315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58173a8c-5532-49c5-918e-d7f112b70659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:28.942918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58173a8c-5532-49c5-918e-d7f112b70659 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:28.943212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58173a8c-5532-49c5-918e-d7f112b70659 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:28.943486 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58173a8c-5532-49c5-918e-d7f112b70659 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:28.944207 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58173a8c-5532-49c5-918e-d7f112b70659 None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=4423f9c79d5048afbd9b0c6eae0bdde1, user.id=5d06a92fb2a5461186204f37e327f5b8) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:28.944514 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58173a8c-5532-49c5-918e-d7f112b70659 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:28.946523 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58173a8c-5532-49c5-918e-d7f112b70659 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:28.959133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 524/1057] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:39:28 2022] GET /identity/v3/role_assignments?user.id=5d06a92fb2a5461186204f37e327f5b8&scope.project.id=4423f9c79d5048afbd9b0c6eae0bdde1&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:29.044612 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc20b6-8dab-4ac3-aeb5-45130fed1c06 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:29.081230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb99cc-5f0b-483d-871b-92ec82f6d4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:29.081851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb99cc-5f0b-483d-871b-92ec82f6d4a3 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:29.082137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb99cc-5f0b-483d-871b-92ec82f6d4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:29.082433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb99cc-5f0b-483d-871b-92ec82f6d4a3 None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:29.083081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb99cc-5f0b-483d-871b-92ec82f6d4a3 None admin] RBAC: Adding query filter params (type=image) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:29.083420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb99cc-5f0b-483d-871b-92ec82f6d4a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:29.084802 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb99cc-5f0b-483d-871b-92ec82f6d4a3 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:29.090175 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 534/1058] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:29.096145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58173a8c-5532-49c5-918e-d7f112b70659 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:29.156516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:29.201619 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58173a8c-5532-49c5-918e-d7f112b70659 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:29.237909 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da2af8-18ed-41ab-a17f-acaebf95ed8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:29.238518 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da2af8-18ed-41ab-a17f-acaebf95ed8d None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:29.238795 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da2af8-18ed-41ab-a17f-acaebf95ed8d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:29.239082 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da2af8-18ed-41ab-a17f-acaebf95ed8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:29.239771 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da2af8-18ed-41ab-a17f-acaebf95ed8d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:29.241126 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da2af8-18ed-41ab-a17f-acaebf95ed8d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:29.323197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 525/1059] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:39:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:39:29.543986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc85d47-63f1-4423-9cb5-fc6cbc7225df None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:29.544338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc85d47-63f1-4423-9cb5-fc6cbc7225df None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:29.544550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc85d47-63f1-4423-9cb5-fc6cbc7225df None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:29.546098 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 535/1060] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:29.551007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59564c65-1757-4d68-8224-6f0b5cd04561 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:29.551324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59564c65-1757-4d68-8224-6f0b5cd04561 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:29.551600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59564c65-1757-4d68-8224-6f0b5cd04561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:29.570527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-59564c65-1757-4d68-8224-6f0b5cd04561 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:29.571768 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-59564c65-1757-4d68-8224-6f0b5cd04561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:29.650239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 526/1061] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:29.922426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc85d47-63f1-4423-9cb5-fc6cbc7225df None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:29.934076 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:30.024443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc85d47-63f1-4423-9cb5-fc6cbc7225df None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:30.061568 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba4528-6701-40cb-80a3-42dd6b2e83a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:30.062310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba4528-6701-40cb-80a3-42dd6b2e83a0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:30.062543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba4528-6701-40cb-80a3-42dd6b2e83a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:30.062735 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba4528-6701-40cb-80a3-42dd6b2e83a0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:30.072953 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba4528-6701-40cb-80a3-42dd6b2e83a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:39:30.077783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-a7ba4528-6701-40cb-80a3-42dd6b2e83a0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:39:30.077783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:30.077783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:30.077783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:30.077783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:30.077783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:30.077783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:30.077783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:30.077783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:30.077783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:30.077783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:30.077783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:30.077783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:39:30.077783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:39:30.077783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:39:30.077783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:39:30.078733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:39:30.078733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:39:30.078733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:39:30.078733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:39:30.078733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:30.078733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:30.078733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:30.078733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:30.078733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:30.078733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:30.078733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:30.078733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:30.078733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:30.078733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:30.078733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:30.078733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:30.079657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:30.079657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:30.079657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:30.079657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:30.079657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:39:30.079657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:39:30.079657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:39:30.079657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:39:30.079657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:39:30.079657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:39:30.079657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:39:30.079657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:30.081098 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 536/1062] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Aug 30 14:39:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:30.084774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59564c65-1757-4d68-8224-6f0b5cd04561 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:30.125100 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:30.187429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59564c65-1757-4d68-8224-6f0b5cd04561 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:30.221521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342e108-d35b-4c00-8fa3-edc27d03a038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:30.222121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342e108-d35b-4c00-8fa3-edc27d03a038 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:30.222402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342e108-d35b-4c00-8fa3-edc27d03a038 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:30.222672 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342e108-d35b-4c00-8fa3-edc27d03a038 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:30.223357 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342e108-d35b-4c00-8fa3-edc27d03a038 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:30.223662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342e108-d35b-4c00-8fa3-edc27d03a038 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:30.225025 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342e108-d35b-4c00-8fa3-edc27d03a038 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:30.234579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 527/1063] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Aug 30 14:39:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:30.240134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba4528-6701-40cb-80a3-42dd6b2e83a0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:30.314226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:30.396000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba4528-6701-40cb-80a3-42dd6b2e83a0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:30.433251 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1889fb-bbab-4d53-bf0d-c205450dd7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:30.433930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1889fb-bbab-4d53-bf0d-c205450dd7d4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:30.434213 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1889fb-bbab-4d53-bf0d-c205450dd7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:30.434485 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1889fb-bbab-4d53-bf0d-c205450dd7d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:30.444782 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1889fb-bbab-4d53-bf0d-c205450dd7d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:30.446289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1889fb-bbab-4d53-bf0d-c205450dd7d4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:30.460756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-ab1889fb-bbab-4d53-bf0d-c205450dd7d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:30.460756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:30.460756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:30.460756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:30.460756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:30.460756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:30.460756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:30.460756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:30.460756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:30.460756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:30.460756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:30.460756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:30.460756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:39:30.460756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:39:30.460756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:39:30.460756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:30.462365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:39:30.468047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:39:30.468047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:30.468047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:30.468047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 537/1064] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:30.468299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342e108-d35b-4c00-8fa3-edc27d03a038 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:30.571585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:30.574743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342e108-d35b-4c00-8fa3-edc27d03a038 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:30.609728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfadb2fe-a12e-48d1-b7cb-affdde416b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:30.610327 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfadb2fe-a12e-48d1-b7cb-affdde416b9d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:30.610604 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfadb2fe-a12e-48d1-b7cb-affdde416b9d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:30.610872 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfadb2fe-a12e-48d1-b7cb-affdde416b9d None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:30.611536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfadb2fe-a12e-48d1-b7cb-affdde416b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:30.611838 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfadb2fe-a12e-48d1-b7cb-affdde416b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:30.613196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfadb2fe-a12e-48d1-b7cb-affdde416b9d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:30.629079 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 528/1065] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:30.630260 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1889fb-bbab-4d53-bf0d-c205450dd7d4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:30.730860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1889fb-bbab-4d53-bf0d-c205450dd7d4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:30.750872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:30.765724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bfeed5-ad13-4b95-9dd7-026e3c691052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:30.769442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfeed5-ad13-4b95-9dd7-026e3c691052 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:30.769442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfeed5-ad13-4b95-9dd7-026e3c691052 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:30.769442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfeed5-ad13-4b95-9dd7-026e3c691052 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:30.775736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bfeed5-ad13-4b95-9dd7-026e3c691052 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:30.777231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bfeed5-ad13-4b95-9dd7-026e3c691052 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:30.788255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-67bfeed5-ad13-4b95-9dd7-026e3c691052 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:39:30.788255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:30.788255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:30.788255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:30.788255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:30.788255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:30.788255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:30.788255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:30.788255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:30.788255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:30.788255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:30.788255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:30.788255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:39:30.788255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:39:30.788255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:39:30.788255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:30.789395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:30.795297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:30.795297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:30.795297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:39:30.795297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:39:30.795297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:39:30.795297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:39:30.795297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:39:30.795297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:39:30.795297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:39:30.795297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:30.795297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 538/1066] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 30 14:39:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:30.795912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfadb2fe-a12e-48d1-b7cb-affdde416b9d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:30.863418 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e951806-2b71-4bfc-a10d-0f08668eb48a None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:30.863762 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e951806-2b71-4bfc-a10d-0f08668eb48a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:30.863969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e951806-2b71-4bfc-a10d-0f08668eb48a None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:30.865448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 539/1067] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:39:30.870136 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08eebf-f109-4ad1-8242-2a25c723f7a2 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:30.870419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08eebf-f109-4ad1-8242-2a25c723f7a2 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:30.870694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08eebf-f109-4ad1-8242-2a25c723f7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:30.889857 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-8b08eebf-f109-4ad1-8242-2a25c723f7a2 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:30.891042 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8b08eebf-f109-4ad1-8242-2a25c723f7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:30.901071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfadb2fe-a12e-48d1-b7cb-affdde416b9d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:30.937231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa337af-9688-46bd-a74d-38f1eb53ec53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:30.937896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa337af-9688-46bd-a74d-38f1eb53ec53 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:30.938173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa337af-9688-46bd-a74d-38f1eb53ec53 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:30.938442 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa337af-9688-46bd-a74d-38f1eb53ec53 None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:30.939196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa337af-9688-46bd-a74d-38f1eb53ec53 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:30.939504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa337af-9688-46bd-a74d-38f1eb53ec53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:30.941184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa337af-9688-46bd-a74d-38f1eb53ec53 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:30.960459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 529/1068] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 30 14:39:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:30.962203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 540/1069] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:39:30.965983 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa337af-9688-46bd-a74d-38f1eb53ec53 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:30.985662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:31.066210 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa337af-9688-46bd-a74d-38f1eb53ec53 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:31.100139 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f963e6-7bfb-4f76-9a64-81c544991c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:31.100749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f963e6-7bfb-4f76-9a64-81c544991c7d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:31.101023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f963e6-7bfb-4f76-9a64-81c544991c7d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:31.101290 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f963e6-7bfb-4f76-9a64-81c544991c7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:31.111293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f963e6-7bfb-4f76-9a64-81c544991c7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:31.112723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f963e6-7bfb-4f76-9a64-81c544991c7d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:31.127737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-94f963e6-7bfb-4f76-9a64-81c544991c7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:31.127737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:31.127737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:31.127737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:31.127737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:31.127737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:31.127737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:31.127737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:31.127737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:31.127737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:31.127737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:31.127737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:31.127737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:39:31.127737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:39:31.127737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:39:31.127737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:31.128692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:39:31.129595 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:39:31.129595 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:31.129595 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:31.131535 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 530/1070] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:31.134957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08eebf-f109-4ad1-8242-2a25c723f7a2 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:31.228332 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f963e6-7bfb-4f76-9a64-81c544991c7d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:31.243721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08eebf-f109-4ad1-8242-2a25c723f7a2 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:31.254772 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:31.279934 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e84e99-4918-4a5e-9fd2-7aa7670c14e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:31.280548 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e84e99-4918-4a5e-9fd2-7aa7670c14e3 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:31.280833 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e84e99-4918-4a5e-9fd2-7aa7670c14e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:31.281435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e84e99-4918-4a5e-9fd2-7aa7670c14e3 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:31.282233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e84e99-4918-4a5e-9fd2-7aa7670c14e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:31.285998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e84e99-4918-4a5e-9fd2-7aa7670c14e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:31.285998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e84e99-4918-4a5e-9fd2-7aa7670c14e3 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:31.295123 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 541/1071] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:31.296276 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308598c-cdb5-4587-8958-2b91cfd19ae1 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:31.296555 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308598c-cdb5-4587-8958-2b91cfd19ae1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:31.296823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308598c-cdb5-4587-8958-2b91cfd19ae1 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:31.298204 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 542/1072] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:31.300744 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308598c-cdb5-4587-8958-2b91cfd19ae1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:31.308572 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:31.340837 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f963e6-7bfb-4f76-9a64-81c544991c7d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:31.373539 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63febfa8-3c4c-4768-8697-ecf7155cdbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:31.374156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63febfa8-3c4c-4768-8697-ecf7155cdbbf None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:31.374437 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63febfa8-3c4c-4768-8697-ecf7155cdbbf None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:31.374708 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63febfa8-3c4c-4768-8697-ecf7155cdbbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:31.385177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63febfa8-3c4c-4768-8697-ecf7155cdbbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:39:31.390429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-63febfa8-3c4c-4768-8697-ecf7155cdbbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:39:31.390429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:31.390429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:31.390429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:31.390429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:31.390429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:31.390429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:31.390429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:31.390429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:31.390429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:31.390429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:31.390429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:31.390429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:39:31.390429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:39:31.390429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:39:31.390429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:39:31.391420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:39:31.391420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:39:31.391420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:39:31.391420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:39:31.391420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:31.391420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:31.391420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:31.391420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:31.391420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:31.391420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:31.391420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:31.391420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:31.391420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:31.391420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:31.391420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:31.391420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:31.396199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:31.396199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:31.396199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:31.396199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:31.396199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:39:31.396199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:39:31.396199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:39:31.396199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:39:31.396199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:39:31.396199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:39:31.396199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:39:31.396199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:31.396199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 531/1073] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:39:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:31.396199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273558b2-21ba-4737-a8ee-907a5cc64c04 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:31.396956 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273558b2-21ba-4737-a8ee-907a5cc64c04 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:31.396956 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273558b2-21ba-4737-a8ee-907a5cc64c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:31.404020 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308598c-cdb5-4587-8958-2b91cfd19ae1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:31.411490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-273558b2-21ba-4737-a8ee-907a5cc64c04 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:31.412687 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-273558b2-21ba-4737-a8ee-907a5cc64c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:31.438173 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdd292-cafd-46c9-9bee-4b112d4b447a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:31.438788 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdd292-cafd-46c9-9bee-4b112d4b447a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:31.439059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdd292-cafd-46c9-9bee-4b112d4b447a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:31.439358 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdd292-cafd-46c9-9bee-4b112d4b447a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:31.446249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdd292-cafd-46c9-9bee-4b112d4b447a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:31.447707 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdd292-cafd-46c9-9bee-4b112d4b447a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:31.458427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-cbbdd292-cafd-46c9-9bee-4b112d4b447a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:31.458427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:31.458427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:31.458427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:31.458427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:31.458427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:31.458427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:31.458427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:31.458427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:31.458427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:31.458427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:31.458427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:31.458427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:39:31.458427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:39:31.458427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:31.459325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:31.460234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:39:31.460234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:39:31.460234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:39:31.460234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:39:31.460234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:39:31.460234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:39:31.460234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:31.460234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:31.460234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 543/1074] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:39:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:31.460234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdd292-cafd-46c9-9bee-4b112d4b447a None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:31.481806 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 532/1075] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:31.482621 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273558b2-21ba-4737-a8ee-907a5cc64c04 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:31.498614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:31.546503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:31.572757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdd292-cafd-46c9-9bee-4b112d4b447a None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:31.588731 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273558b2-21ba-4737-a8ee-907a5cc64c04 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:31.608671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588cf356-5a99-44f7-8703-ddaeabdb4546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:31.609625 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cf356-5a99-44f7-8703-ddaeabdb4546 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:31.610106 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cf356-5a99-44f7-8703-ddaeabdb4546 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:31.610577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588cf356-5a99-44f7-8703-ddaeabdb4546 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:31.611448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588cf356-5a99-44f7-8703-ddaeabdb4546 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:31.612038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588cf356-5a99-44f7-8703-ddaeabdb4546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:31.613700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588cf356-5a99-44f7-8703-ddaeabdb4546 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:31.622722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85539d9b-a369-47fc-bde5-902fa5cfb0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:31.623343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85539d9b-a369-47fc-bde5-902fa5cfb0e5 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:31.623537 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 544/1076] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:39:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:31.623638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85539d9b-a369-47fc-bde5-902fa5cfb0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:31.623869 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85539d9b-a369-47fc-bde5-902fa5cfb0e5 None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:31.624552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85539d9b-a369-47fc-bde5-902fa5cfb0e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:31.624853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85539d9b-a369-47fc-bde5-902fa5cfb0e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:31.626546 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85539d9b-a369-47fc-bde5-902fa5cfb0e5 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:31.629049 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588cf356-5a99-44f7-8703-ddaeabdb4546 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:31.640292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 533/1077] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:39:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:31.646015 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85539d9b-a369-47fc-bde5-902fa5cfb0e5 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:31.727851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:31.737263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588cf356-5a99-44f7-8703-ddaeabdb4546 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:31.744800 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85539d9b-a369-47fc-bde5-902fa5cfb0e5 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:31.766294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:31.774257 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20cb05-388d-497f-ab26-0e021b681296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:31.774945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20cb05-388d-497f-ab26-0e021b681296 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:31.778412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20cb05-388d-497f-ab26-0e021b681296 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:31.778412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20cb05-388d-497f-ab26-0e021b681296 None admin] PATH_INFO: `/v3/users/5d06a92fb2a5461186204f37e327f5b8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:31.778412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20cb05-388d-497f-ab26-0e021b681296 None admin] RBAC: Authorizing `identity:get_user(user_id=5d06a92fb2a5461186204f37e327f5b8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:31.778412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20cb05-388d-497f-ab26-0e021b681296 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:31.779670 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 545/1078] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:39:31 2022] GET /identity/v3/users/5d06a92fb2a5461186204f37e327f5b8 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:31.780252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1ca60-ecc8-4d3c-9d32-943a5be597c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:31.780378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20cb05-388d-497f-ab26-0e021b681296 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:31.780901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1ca60-ecc8-4d3c-9d32-943a5be597c5 None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:31.781173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1ca60-ecc8-4d3c-9d32-943a5be597c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:31.781437 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1ca60-ecc8-4d3c-9d32-943a5be597c5 None admin] PATH_INFO: `/v3/projects/1147574d59c7404baf76a24149684f6e/users/5b6e4c5bf2a54b28a55e5ae74b90ab4f/roles/bcecc24215984de9a00652caa5433ffe` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:31.783585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1ca60-ecc8-4d3c-9d32-943a5be597c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=1147574d59c7404baf76a24149684f6e, user_id=5b6e4c5bf2a54b28a55e5ae74b90ab4f, role_id=bcecc24215984de9a00652caa5433ffe)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:31.784983 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1ca60-ecc8-4d3c-9d32-943a5be597c5 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:31.852636 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 534/1079] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:39:31 2022] PUT /identity/v3/projects/1147574d59c7404baf76a24149684f6e/users/5b6e4c5bf2a54b28a55e5ae74b90ab4f/roles/bcecc24215984de9a00652caa5433ffe => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:39:31.853427 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1ca60-ecc8-4d3c-9d32-943a5be597c5 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:31.884437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20cb05-388d-497f-ab26-0e021b681296 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:31.923399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4c95d-35f1-491f-a3da-cd708387df3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:31.924020 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4c95d-35f1-491f-a3da-cd708387df3c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:31.924298 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4c95d-35f1-491f-a3da-cd708387df3c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:31.924575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4c95d-35f1-491f-a3da-cd708387df3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:31.958710 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:31.971007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1ca60-ecc8-4d3c-9d32-943a5be597c5 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:32.004120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4c95d-35f1-491f-a3da-cd708387df3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:32.005575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4c95d-35f1-491f-a3da-cd708387df3c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:32.014560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084b5e5-9ffd-4036-a427-9ae011a692eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:32.015465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084b5e5-9ffd-4036-a427-9ae011a692eb None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:32.015743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084b5e5-9ffd-4036-a427-9ae011a692eb None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:32.016010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084b5e5-9ffd-4036-a427-9ae011a692eb None admin] PATH_INFO: `/v3/projects/4423f9c79d5048afbd9b0c6eae0bdde1` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:32.017204 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084b5e5-9ffd-4036-a427-9ae011a692eb None admin] RBAC: Authorizing `identity:get_project(project_id=4423f9c79d5048afbd9b0c6eae0bdde1)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:32.018557 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084b5e5-9ffd-4036-a427-9ae011a692eb None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:32.020283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 535/1080] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:39:31 2022] GET /identity/v3/projects/4423f9c79d5048afbd9b0c6eae0bdde1 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:32.026538 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084b5e5-9ffd-4036-a427-9ae011a692eb None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:32.078914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:32.111157 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 546/1081] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:39:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:39:32.116606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4c95d-35f1-491f-a3da-cd708387df3c None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:32.132347 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084b5e5-9ffd-4036-a427-9ae011a692eb None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:32.172685 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57193e37-41bc-4797-b7df-fa27fb3eabe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:32.173337 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57193e37-41bc-4797-b7df-fa27fb3eabe2 None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:32.173613 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57193e37-41bc-4797-b7df-fa27fb3eabe2 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:32.173880 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57193e37-41bc-4797-b7df-fa27fb3eabe2 None admin] PATH_INFO: `/v3/projects/4423f9c79d5048afbd9b0c6eae0bdde1/users/5d06a92fb2a5461186204f37e327f5b8/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:32.176514 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57193e37-41bc-4797-b7df-fa27fb3eabe2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4423f9c79d5048afbd9b0c6eae0bdde1, user_id=5d06a92fb2a5461186204f37e327f5b8, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:32.177942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57193e37-41bc-4797-b7df-fa27fb3eabe2 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:32.179491 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:32.225907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4c95d-35f1-491f-a3da-cd708387df3c None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:32.245600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 536/1082] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:39:32 2022] PUT /identity/v3/projects/4423f9c79d5048afbd9b0c6eae0bdde1/users/5d06a92fb2a5461186204f37e327f5b8/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:39:32.275627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1cdd1-d8b5-44d0-aeb4-046fe1e97f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:32.276184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1cdd1-d8b5-44d0-aeb4-046fe1e97f71 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:32.276459 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1cdd1-d8b5-44d0-aeb4-046fe1e97f71 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:32.276723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1cdd1-d8b5-44d0-aeb4-046fe1e97f71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:32.288384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1cdd1-d8b5-44d0-aeb4-046fe1e97f71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:32.289889 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1cdd1-d8b5-44d0-aeb4-046fe1e97f71 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:32.306399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-caf1cdd1-d8b5-44d0-aeb4-046fe1e97f71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:32.306399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:32.306399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:32.306399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:32.306399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:32.306399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:32.306399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:32.306399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:32.306399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:32.306399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:32.306399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:32.306399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:32.306399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:39:32.306399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:39:32.306399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:39:32.306399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:32.310691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:39:32.311823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:39:32.311823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:32.311823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:32.311823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 547/1083] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:32.313556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57193e37-41bc-4797-b7df-fa27fb3eabe2 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:32.424644 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57193e37-41bc-4797-b7df-fa27fb3eabe2 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:32.434811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:32.459609 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b244f-1d1d-4fdd-b999-a7d8f4a9dc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:32.460213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b244f-1d1d-4fdd-b999-a7d8f4a9dc0d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:32.460495 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b244f-1d1d-4fdd-b999-a7d8f4a9dc0d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:32.460763 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b244f-1d1d-4fdd-b999-a7d8f4a9dc0d None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:32.461441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b244f-1d1d-4fdd-b999-a7d8f4a9dc0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:32.461829 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b244f-1d1d-4fdd-b999-a7d8f4a9dc0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:32.463205 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b244f-1d1d-4fdd-b999-a7d8f4a9dc0d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:32.475030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 537/1084] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:33.764588 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f4177-58f4-482b-b973-9747c5c113fc None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:33.765110 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f4177-58f4-482b-b973-9747c5c113fc None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:33.765202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f4177-58f4-482b-b973-9747c5c113fc None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:33.766621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 548/1085] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:33.771662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a5e76-1cd3-494c-90c6-37de23536892 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:33.772057 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a5e76-1cd3-494c-90c6-37de23536892 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:33.772340 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a5e76-1cd3-494c-90c6-37de23536892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:33.791589 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-0f2a5e76-1cd3-494c-90c6-37de23536892 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:33.792778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0f2a5e76-1cd3-494c-90c6-37de23536892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:33.858147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 538/1086] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:34.123401 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f4177-58f4-482b-b973-9747c5c113fc None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:34.147258 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad395b-c218-4c86-8fd0-e40dac6cba35 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:34.147550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad395b-c218-4c86-8fd0-e40dac6cba35 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:34.148087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad395b-c218-4c86-8fd0-e40dac6cba35 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:34.149913 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 539/1087] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:39:34.154927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8556f-4fcf-4bec-a3b0-738519ce7eae None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:34.155244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8556f-4fcf-4bec-a3b0-738519ce7eae None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:34.155524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8556f-4fcf-4bec-a3b0-738519ce7eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:34.174592 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-40d8556f-4fcf-4bec-a3b0-738519ce7eae None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:34.175822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-40d8556f-4fcf-4bec-a3b0-738519ce7eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:34.227625 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f4177-58f4-482b-b973-9747c5c113fc None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:34.243960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:34.245713 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 540/1088] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:39:34.269773 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6395b-1a9f-45cc-988d-364b7f6cd164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:34.270403 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6395b-1a9f-45cc-988d-364b7f6cd164 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:34.270681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6395b-1a9f-45cc-988d-364b7f6cd164 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:34.270946 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6395b-1a9f-45cc-988d-364b7f6cd164 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:34.282586 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6395b-1a9f-45cc-988d-364b7f6cd164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:39:34.287804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-3cb6395b-1a9f-45cc-988d-364b7f6cd164 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:39:34.287804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:34.287804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:34.287804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:34.287804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:34.287804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:34.287804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:34.287804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:34.287804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:34.287804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:34.287804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:34.287804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:34.287804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:39:34.287804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:39:34.287804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:39:34.287804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:39:34.289364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:39:34.289364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:39:34.289364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:39:34.289364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:39:34.289364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:34.289364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:34.289364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:34.289364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:34.289364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:34.289364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:34.289364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:34.289364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:34.289364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:34.289364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:34.289364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:34.289364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:34.290336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:34.290336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:34.290336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:34.290336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:34.290336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:39:34.290336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:39:34.290336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:39:34.290336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:39:34.290336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:39:34.290336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:39:34.290336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:39:34.290336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:34.290336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 549/1089] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Aug 30 14:39:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:34.298383 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8556f-4fcf-4bec-a3b0-738519ce7eae None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:34.329348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:34.401024 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8556f-4fcf-4bec-a3b0-738519ce7eae None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:34.433368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef681a9-95b9-4462-b377-dde1656bd525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:34.433971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef681a9-95b9-4462-b377-dde1656bd525 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:34.434250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef681a9-95b9-4462-b377-dde1656bd525 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:34.434525 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef681a9-95b9-4462-b377-dde1656bd525 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:34.435219 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef681a9-95b9-4462-b377-dde1656bd525 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:34.435522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef681a9-95b9-4462-b377-dde1656bd525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:34.436876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef681a9-95b9-4462-b377-dde1656bd525 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:34.444541 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 541/1090] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Aug 30 14:39:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:34.450768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6395b-1a9f-45cc-988d-364b7f6cd164 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:34.526145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef681a9-95b9-4462-b377-dde1656bd525 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:34.549075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6395b-1a9f-45cc-988d-364b7f6cd164 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:34.583332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e2c25-923d-4b00-8555-e7759294e145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:34.583960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e2c25-923d-4b00-8555-e7759294e145 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:34.584241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e2c25-923d-4b00-8555-e7759294e145 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:34.584521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e2c25-923d-4b00-8555-e7759294e145 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:34.594752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e2c25-923d-4b00-8555-e7759294e145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:34.596241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e2c25-923d-4b00-8555-e7759294e145 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:34.611083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-0c6e2c25-923d-4b00-8555-e7759294e145 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:34.611083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:34.611083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:34.611083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:34.611083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:34.611083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:34.611083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:34.611083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:34.611083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:34.611083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:34.611083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:34.611083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:34.611083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:39:34.611083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:39:34.611083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:39:34.611083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:34.612215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:39:34.613108 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:39:34.613108 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:34.613108 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:34.613108 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 550/1091] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:34.616229 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:34.618098 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e2c25-923d-4b00-8555-e7759294e145 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:34.625981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef681a9-95b9-4462-b377-dde1656bd525 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:34.643442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:34.661026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662362b6-cd95-49eb-b695-948160ac0144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:34.661588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662362b6-cd95-49eb-b695-948160ac0144 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:34.661885 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662362b6-cd95-49eb-b695-948160ac0144 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:34.662160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662362b6-cd95-49eb-b695-948160ac0144 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:34.672214 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662362b6-cd95-49eb-b695-948160ac0144 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:39:34.676899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-662362b6-cd95-49eb-b695-948160ac0144 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:39:34.676899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:34.676899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:34.676899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:34.676899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:34.676899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:34.676899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:34.676899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:34.676899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:34.676899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:34.676899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:34.676899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:34.676899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:39:34.676899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:39:34.676899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:39:34.676899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:39:34.677940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:39:34.677940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:39:34.677940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:39:34.677940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:39:34.677940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:34.677940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:34.677940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:34.677940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:34.677940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:34.677940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:34.677940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:34.677940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:34.677940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:34.677940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:34.677940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:34.677940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:34.678785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:34.678785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:34.678785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:34.678785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:34.678785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:39:34.678785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:39:34.678785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:39:34.678785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:39:34.678785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:39:34.678785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:39:34.678785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:39:34.678785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:34.683486 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 542/1092] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:39:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:34.684290 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662362b6-cd95-49eb-b695-948160ac0144 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:34.717056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e2c25-923d-4b00-8555-e7759294e145 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:34.750175 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17ad30-a043-4818-92f8-eb25d730672c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:34.750767 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17ad30-a043-4818-92f8-eb25d730672c None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:34.751046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17ad30-a043-4818-92f8-eb25d730672c None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:34.751341 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17ad30-a043-4818-92f8-eb25d730672c None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:34.751973 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17ad30-a043-4818-92f8-eb25d730672c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:34.752270 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17ad30-a043-4818-92f8-eb25d730672c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:34.753624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17ad30-a043-4818-92f8-eb25d730672c None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:34.765620 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 551/1093] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:34.771179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17ad30-a043-4818-92f8-eb25d730672c None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:34.782053 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662362b6-cd95-49eb-b695-948160ac0144 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:34.815659 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72033900-7fb6-408b-87d9-4d29ae65c08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:34.816248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72033900-7fb6-408b-87d9-4d29ae65c08d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:34.816521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72033900-7fb6-408b-87d9-4d29ae65c08d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:34.816791 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72033900-7fb6-408b-87d9-4d29ae65c08d None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:34.817435 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72033900-7fb6-408b-87d9-4d29ae65c08d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:34.817729 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72033900-7fb6-408b-87d9-4d29ae65c08d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:34.819075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72033900-7fb6-408b-87d9-4d29ae65c08d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:34.827733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 543/1094] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:39:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:34.833145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72033900-7fb6-408b-87d9-4d29ae65c08d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:34.847213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:34.852904 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:34.868585 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17ad30-a043-4818-92f8-eb25d730672c None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:34.901204 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff03fa-7c6d-4120-9e59-8c6489ada3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:34.901812 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff03fa-7c6d-4120-9e59-8c6489ada3d1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:34.902085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff03fa-7c6d-4120-9e59-8c6489ada3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:34.902355 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff03fa-7c6d-4120-9e59-8c6489ada3d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:34.909247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff03fa-7c6d-4120-9e59-8c6489ada3d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:34.910665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff03fa-7c6d-4120-9e59-8c6489ada3d1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:34.922549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-66ff03fa-7c6d-4120-9e59-8c6489ada3d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:39:34.922549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:34.922549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:34.922549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:34.922549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:34.922549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:34.922549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:34.922549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:34.922549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:34.922549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:34.922549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:34.922549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:34.922549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:39:34.922549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:39:34.922549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:39:34.922549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:34.923502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:34.924482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:34.924482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:34.924482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:39:34.924482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:39:34.924482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:39:34.924482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:39:34.924482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:39:34.924482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:39:34.924482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:39:34.924482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:34.924482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 552/1095] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Aug 30 14:39:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:34.929203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff03fa-7c6d-4120-9e59-8c6489ada3d1 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:34.930906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72033900-7fb6-408b-87d9-4d29ae65c08d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:34.965409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a435137-6521-4e57-82be-ae536b24e638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:34.966032 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a435137-6521-4e57-82be-ae536b24e638 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:34.966301 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a435137-6521-4e57-82be-ae536b24e638 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:34.966564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a435137-6521-4e57-82be-ae536b24e638 None admin] PATH_INFO: `/v3/users/5d06a92fb2a5461186204f37e327f5b8` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:34.967757 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a435137-6521-4e57-82be-ae536b24e638 None admin] RBAC: Authorizing `identity:get_user(user_id=5d06a92fb2a5461186204f37e327f5b8)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:34.969163 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a435137-6521-4e57-82be-ae536b24e638 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:34.970720 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 544/1096] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:39:34 2022] GET /identity/v3/users/5d06a92fb2a5461186204f37e327f5b8 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:34.976398 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a435137-6521-4e57-82be-ae536b24e638 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:35.027648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff03fa-7c6d-4120-9e59-8c6489ada3d1 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:35.056048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:35.061259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e1ad4-875b-46c6-a496-faa29348c8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:35.061866 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e1ad4-875b-46c6-a496-faa29348c8b1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:35.062145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e1ad4-875b-46c6-a496-faa29348c8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:35.062440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e1ad4-875b-46c6-a496-faa29348c8b1 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:35.063276 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e1ad4-875b-46c6-a496-faa29348c8b1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:35.063583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e1ad4-875b-46c6-a496-faa29348c8b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:35.065005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e1ad4-875b-46c6-a496-faa29348c8b1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:35.074228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a435137-6521-4e57-82be-ae536b24e638 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:35.081820 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 553/1097] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Aug 30 14:39:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:35.087709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e1ad4-875b-46c6-a496-faa29348c8b1 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:35.090381 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:35.108086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1093a1-9e41-42b2-930d-69a3e9862fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:35.108723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1093a1-9e41-42b2-930d-69a3e9862fcb None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:35.109019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1093a1-9e41-42b2-930d-69a3e9862fcb None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:35.109273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1093a1-9e41-42b2-930d-69a3e9862fcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:35.176226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1093a1-9e41-42b2-930d-69a3e9862fcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:35.177584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1093a1-9e41-42b2-930d-69a3e9862fcb None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:35.184570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e1ad4-875b-46c6-a496-faa29348c8b1 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:35.216845 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de98d2-f1d2-42d2-9c7f-6c138126fdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:35.217438 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de98d2-f1d2-42d2-9c7f-6c138126fdc1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:35.217713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de98d2-f1d2-42d2-9c7f-6c138126fdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:35.217986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de98d2-f1d2-42d2-9c7f-6c138126fdc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:35.256233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:35.278951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 545/1098] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:39:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 14:39:35.284810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1093a1-9e41-42b2-930d-69a3e9862fcb None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:35.288645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de98d2-f1d2-42d2-9c7f-6c138126fdc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:35.290123 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de98d2-f1d2-42d2-9c7f-6c138126fdc1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:35.291958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:35.381717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1093a1-9e41-42b2-930d-69a3e9862fcb None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:35.386212 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 554/1099] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:39:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:39:35.391958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de98d2-f1d2-42d2-9c7f-6c138126fdc1 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:35.425537 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdebeed-ba60-4b9d-b27f-2baf5ef9ada4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:35.426165 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdebeed-ba60-4b9d-b27f-2baf5ef9ada4 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:35.426493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdebeed-ba60-4b9d-b27f-2baf5ef9ada4 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:35.426706 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdebeed-ba60-4b9d-b27f-2baf5ef9ada4 None admin] PATH_INFO: `/v3/projects/4423f9c79d5048afbd9b0c6eae0bdde1` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:35.427833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdebeed-ba60-4b9d-b27f-2baf5ef9ada4 None admin] RBAC: Authorizing `identity:get_project(project_id=4423f9c79d5048afbd9b0c6eae0bdde1)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:35.429250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdebeed-ba60-4b9d-b27f-2baf5ef9ada4 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:35.431421 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 546/1100] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:39:35 2022] GET /identity/v3/projects/4423f9c79d5048afbd9b0c6eae0bdde1 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:35.437336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdebeed-ba60-4b9d-b27f-2baf5ef9ada4 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:35.486277 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:35.494814 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de98d2-f1d2-42d2-9c7f-6c138126fdc1 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:35.496366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:35.528678 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9341b-02f6-4f50-8177-b1db076e48e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:35.529296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9341b-02f6-4f50-8177-b1db076e48e2 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:35.529575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9341b-02f6-4f50-8177-b1db076e48e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:35.529848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9341b-02f6-4f50-8177-b1db076e48e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:35.535563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdebeed-ba60-4b9d-b27f-2baf5ef9ada4 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:35.539677 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9341b-02f6-4f50-8177-b1db076e48e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:35.541154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9341b-02f6-4f50-8177-b1db076e48e2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:35.555604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-24b9341b-02f6-4f50-8177-b1db076e48e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:35.555604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:35.555604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:35.555604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:35.555604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:35.555604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:35.555604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:35.555604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:35.555604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:35.555604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:35.555604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:35.555604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:35.555604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:39:35.555604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:39:35.555604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:39:35.555604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:35.556554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:39:35.557852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:39:35.557852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:39:35.557852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:35.557852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 555/1101] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:39:35.562375 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9341b-02f6-4f50-8177-b1db076e48e2 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:35.573585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd565d-6e92-4f0b-8d6c-5994c8424ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:35.574190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd565d-6e92-4f0b-8d6c-5994c8424ef4 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:35.574468 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd565d-6e92-4f0b-8d6c-5994c8424ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:35.574738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd565d-6e92-4f0b-8d6c-5994c8424ef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:35.575493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd565d-6e92-4f0b-8d6c-5994c8424ef4 None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=4423f9c79d5048afbd9b0c6eae0bdde1, user.id=5d06a92fb2a5461186204f37e327f5b8) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:35.575798 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd565d-6e92-4f0b-8d6c-5994c8424ef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:35.577250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd565d-6e92-4f0b-8d6c-5994c8424ef4 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:35.598349 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 547/1102] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:39:35 2022] GET /identity/v3/role_assignments?user.id=5d06a92fb2a5461186204f37e327f5b8&scope.project.id=4423f9c79d5048afbd9b0c6eae0bdde1&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:35.664678 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9341b-02f6-4f50-8177-b1db076e48e2 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:35.703856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6a58e-6070-4fa9-9f99-8dbea874fb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:35.704389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6a58e-6070-4fa9-9f99-8dbea874fb4f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:35.704665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6a58e-6070-4fa9-9f99-8dbea874fb4f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:35.704940 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6a58e-6070-4fa9-9f99-8dbea874fb4f None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:35.705594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6a58e-6070-4fa9-9f99-8dbea874fb4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:35.705893 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6a58e-6070-4fa9-9f99-8dbea874fb4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:35.707303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6a58e-6070-4fa9-9f99-8dbea874fb4f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:35.720089 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 556/1103] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:35.725758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd565d-6e92-4f0b-8d6c-5994c8424ef4 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:35.826582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd565d-6e92-4f0b-8d6c-5994c8424ef4 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:35.839227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:35.860055 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df201b-d756-41ae-9def-047c6bc963d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:35.860681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df201b-d756-41ae-9def-047c6bc963d6 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:35.860960 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df201b-d756-41ae-9def-047c6bc963d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:35.861230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df201b-d756-41ae-9def-047c6bc963d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:35.868405 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df201b-d756-41ae-9def-047c6bc963d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:35.869968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df201b-d756-41ae-9def-047c6bc963d6 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:35.881469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-40df201b-d756-41ae-9def-047c6bc963d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:35.881469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:35.881469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:35.881469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:35.881469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:35.881469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:35.881469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:35.881469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:35.881469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:35.881469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:35.881469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:35.881469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:35.881469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:39:35.881469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:39:35.881469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:35.882511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:35.883786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:39:35.883786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:39:35.883786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:39:35.883786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:39:35.883786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:39:35.883786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:39:35.883786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:39:35.883786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:39:35.883786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 548/1104] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:39:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:35.888510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6a58e-6070-4fa9-9f99-8dbea874fb4f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:35.904520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:35.987385 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6a58e-6070-4fa9-9f99-8dbea874fb4f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:36.020848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bf06c-ce4c-4d75-82c7-da9bbdaf80c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:36.021446 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bf06c-ce4c-4d75-82c7-da9bbdaf80c3 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:36.021723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bf06c-ce4c-4d75-82c7-da9bbdaf80c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:36.021994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bf06c-ce4c-4d75-82c7-da9bbdaf80c3 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:36.022929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2bf06c-ce4c-4d75-82c7-da9bbdaf80c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:36.023294 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2bf06c-ce4c-4d75-82c7-da9bbdaf80c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:36.024718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2bf06c-ce4c-4d75-82c7-da9bbdaf80c3 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:36.036194 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 557/1105] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:39:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:36.042649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df201b-d756-41ae-9def-047c6bc963d6 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:36.143816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df201b-d756-41ae-9def-047c6bc963d6 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:36.178216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b39912-355c-45c2-af58-bdf8e71fadea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:36.178815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b39912-355c-45c2-af58-bdf8e71fadea None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:36.179088 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b39912-355c-45c2-af58-bdf8e71fadea None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:36.179374 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b39912-355c-45c2-af58-bdf8e71fadea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:36.180148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b39912-355c-45c2-af58-bdf8e71fadea None admin] RBAC: Adding query filter params (role.id=bcecc24215984de9a00652caa5433ffe, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=5b6e4c5bf2a54b28a55e5ae74b90ab4f) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:36.180453 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b39912-355c-45c2-af58-bdf8e71fadea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:36.181864 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b39912-355c-45c2-af58-bdf8e71fadea None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:36.187548 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 549/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:39:36 2022] GET /identity/v3/role_assignments?user.id=5b6e4c5bf2a54b28a55e5ae74b90ab4f&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=bcecc24215984de9a00652caa5433ffe => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:37.491432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadee8b-6c64-40d3-b41b-91d5ea750aa3 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:37.491993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadee8b-6c64-40d3-b41b-91d5ea750aa3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:37.491993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadee8b-6c64-40d3-b41b-91d5ea750aa3 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:37.493385 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 558/1107] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:37.498764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801b85b-60b7-40c4-9964-767f3f6af977 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:37.499185 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801b85b-60b7-40c4-9964-767f3f6af977 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:37.499350 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801b85b-60b7-40c4-9964-767f3f6af977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:37.517599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-4801b85b-60b7-40c4-9964-767f3f6af977 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:37.518718 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-4801b85b-60b7-40c4-9964-767f3f6af977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:37.546616 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:37.582785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 550/1108] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:37.851644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadee8b-6c64-40d3-b41b-91d5ea750aa3 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:37.926884 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:37.952209 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadee8b-6c64-40d3-b41b-91d5ea750aa3 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:37.983863 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3eab91-eee3-452e-b4c7-37dde10ad4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:37.984468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3eab91-eee3-452e-b4c7-37dde10ad4eb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:37.984741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3eab91-eee3-452e-b4c7-37dde10ad4eb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:37.985004 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3eab91-eee3-452e-b4c7-37dde10ad4eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:37.986092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3eab91-eee3-452e-b4c7-37dde10ad4eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:37.987532 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3eab91-eee3-452e-b4c7-37dde10ad4eb None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:37.989055 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 559/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:37.995142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801b85b-60b7-40c4-9964-767f3f6af977 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:38.089554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801b85b-60b7-40c4-9964-767f3f6af977 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:38.122517 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f494eb3-8acc-4c6c-a5b0-4e3dd5f1b903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:38.123043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f494eb3-8acc-4c6c-a5b0-4e3dd5f1b903 None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:38.123342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f494eb3-8acc-4c6c-a5b0-4e3dd5f1b903 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:38.123610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f494eb3-8acc-4c6c-a5b0-4e3dd5f1b903 None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:38.124253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f494eb3-8acc-4c6c-a5b0-4e3dd5f1b903 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:38.125653 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f494eb3-8acc-4c6c-a5b0-4e3dd5f1b903 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:38.229780 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 551/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:39:37 2022] POST /identity/v3/users => generated 286 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:39:40.170780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76376c6-f707-45f4-99ca-b4be090cba6c None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:40.171331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76376c6-f707-45f4-99ca-b4be090cba6c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:40.171331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76376c6-f707-45f4-99ca-b4be090cba6c None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:40.172679 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 560/1111] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:40.177690 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d09b0-aa9e-40b8-acf3-8a8979df580d None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:40.178051 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d09b0-aa9e-40b8-acf3-8a8979df580d None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:40.178242 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d09b0-aa9e-40b8-acf3-8a8979df580d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:40.196006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-0e9d09b0-aa9e-40b8-acf3-8a8979df580d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:40.197188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0e9d09b0-aa9e-40b8-acf3-8a8979df580d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:40.213660 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:40.263240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 552/1112] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:40.534829 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76376c6-f707-45f4-99ca-b4be090cba6c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:40.633207 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76376c6-f707-45f4-99ca-b4be090cba6c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:40.666130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be16464-f41b-4872-bb86-11553c302fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:40.666741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be16464-f41b-4872-bb86-11553c302fa8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:40.667021 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be16464-f41b-4872-bb86-11553c302fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:40.667317 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be16464-f41b-4872-bb86-11553c302fa8 None admin] PATH_INFO: `/v3/roles/6fd5089acea448219d4c0a9a9424a3af` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:40.668692 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be16464-f41b-4872-bb86-11553c302fa8 None admin] RBAC: Authorizing `identity:get_role(role_id=6fd5089acea448219d4c0a9a9424a3af)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:40.670055 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be16464-f41b-4872-bb86-11553c302fa8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:40.671396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 561/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:39:40 2022] GET /identity/v3/roles/6fd5089acea448219d4c0a9a9424a3af => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:40.677230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d09b0-aa9e-40b8-acf3-8a8979df580d None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:40.773129 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d09b0-aa9e-40b8-acf3-8a8979df580d None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:40.805752 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fe71b-f604-43f5-9ad2-6ddaa1f69c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:40.806313 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fe71b-f604-43f5-9ad2-6ddaa1f69c84 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:40.806589 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fe71b-f604-43f5-9ad2-6ddaa1f69c84 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:40.806861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fe71b-f604-43f5-9ad2-6ddaa1f69c84 None admin] PATH_INFO: `/v3/users/1eaed6d6b879446fb4ac79eba21dad47` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:40.819340 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94fe71b-f604-43f5-9ad2-6ddaa1f69c84 None admin] RBAC: Authorizing `identity:get_user(user_id=1eaed6d6b879446fb4ac79eba21dad47)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:40.821341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94fe71b-f604-43f5-9ad2-6ddaa1f69c84 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:40.822887 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 553/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:39:40 2022] GET /identity/v3/users/1eaed6d6b879446fb4ac79eba21dad47 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:40.828549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be16464-f41b-4872-bb86-11553c302fa8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:40.840421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:40.923655 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be16464-f41b-4872-bb86-11553c302fa8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:40.956533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1582e-4ce0-482f-ab36-420a4ab114d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:40.957116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1582e-4ce0-482f-ab36-420a4ab114d5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:40.957394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1582e-4ce0-482f-ab36-420a4ab114d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:40.957663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1582e-4ce0-482f-ab36-420a4ab114d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:41.021052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1582e-4ce0-482f-ab36-420a4ab114d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:41.022734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1582e-4ce0-482f-ab36-420a4ab114d5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:41.033113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:41.116592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 562/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:39:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:39:41.122938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fe71b-f604-43f5-9ad2-6ddaa1f69c84 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:41.134769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:41.217740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fe71b-f604-43f5-9ad2-6ddaa1f69c84 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:41.251349 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d061ab-cad5-4765-91ce-98038ebc0d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:41.251963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d061ab-cad5-4765-91ce-98038ebc0d1f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:41.252233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d061ab-cad5-4765-91ce-98038ebc0d1f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:41.252495 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d061ab-cad5-4765-91ce-98038ebc0d1f None admin] PATH_INFO: `/v3/projects/4423f9c79d5048afbd9b0c6eae0bdde1` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:41.253575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d061ab-cad5-4765-91ce-98038ebc0d1f None admin] RBAC: Authorizing `identity:get_project(project_id=4423f9c79d5048afbd9b0c6eae0bdde1)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:41.254980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d061ab-cad5-4765-91ce-98038ebc0d1f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:41.256593 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 554/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:39:41 2022] GET /identity/v3/projects/4423f9c79d5048afbd9b0c6eae0bdde1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:41.262700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1582e-4ce0-482f-ab36-420a4ab114d5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:41.359787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1582e-4ce0-482f-ab36-420a4ab114d5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:41.370543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:41.392337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0e1fa-3d7d-4091-88ed-ab8d2b357099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:41.392943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0e1fa-3d7d-4091-88ed-ab8d2b357099 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:41.393254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0e1fa-3d7d-4091-88ed-ab8d2b357099 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:41.393524 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0e1fa-3d7d-4091-88ed-ab8d2b357099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:41.394239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0e1fa-3d7d-4091-88ed-ab8d2b357099 None admin] RBAC: Adding query filter params (role.id=6fd5089acea448219d4c0a9a9424a3af, scope.project.id=4423f9c79d5048afbd9b0c6eae0bdde1, user.id=1eaed6d6b879446fb4ac79eba21dad47) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:41.394567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0e1fa-3d7d-4091-88ed-ab8d2b357099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:41.396012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0e1fa-3d7d-4091-88ed-ab8d2b357099 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:41.402422 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 563/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:39:41 2022] GET /identity/v3/role_assignments?user.id=1eaed6d6b879446fb4ac79eba21dad47&scope.project.id=4423f9c79d5048afbd9b0c6eae0bdde1&role.id=6fd5089acea448219d4c0a9a9424a3af => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:43.259694 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24f22c-ff2f-484f-b51d-a548e7132f72 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:43.260245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24f22c-ff2f-484f-b51d-a548e7132f72 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:43.260245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24f22c-ff2f-484f-b51d-a548e7132f72 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:43.261570 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 555/1118] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:43.266871 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef880a-f825-4af4-a3f0-538549dde2d9 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:43.267263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef880a-f825-4af4-a3f0-538549dde2d9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:43.267455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef880a-f825-4af4-a3f0-538549dde2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:43.285568 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-d2ef880a-f825-4af4-a3f0-538549dde2d9 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:43.286760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d2ef880a-f825-4af4-a3f0-538549dde2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:43.354915 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 564/1119] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:39:43.620656 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24f22c-ff2f-484f-b51d-a548e7132f72 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:43.698331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:43.720244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24f22c-ff2f-484f-b51d-a548e7132f72 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:43.753034 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930cbb1-7706-44e2-9ff8-acb45da2c6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:43.753647 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930cbb1-7706-44e2-9ff8-acb45da2c6bd None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:43.753927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930cbb1-7706-44e2-9ff8-acb45da2c6bd None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:43.754200 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930cbb1-7706-44e2-9ff8-acb45da2c6bd None admin] PATH_INFO: `/v3/roles/6fd5089acea448219d4c0a9a9424a3af` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:43.755603 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930cbb1-7706-44e2-9ff8-acb45da2c6bd None admin] RBAC: Authorizing `identity:get_role(role_id=6fd5089acea448219d4c0a9a9424a3af)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:43.756976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930cbb1-7706-44e2-9ff8-acb45da2c6bd None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:43.758303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 556/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:39:43 2022] GET /identity/v3/roles/6fd5089acea448219d4c0a9a9424a3af => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:43.764125 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef880a-f825-4af4-a3f0-538549dde2d9 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:43.858496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef880a-f825-4af4-a3f0-538549dde2d9 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:43.860724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:43.890322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345602c6-5f01-4169-aa39-0a039aa218a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:43.890869 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345602c6-5f01-4169-aa39-0a039aa218a2 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:43.891170 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345602c6-5f01-4169-aa39-0a039aa218a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:43.891442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345602c6-5f01-4169-aa39-0a039aa218a2 None admin] PATH_INFO: `/v3/users/1eaed6d6b879446fb4ac79eba21dad47` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:43.892541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345602c6-5f01-4169-aa39-0a039aa218a2 None admin] RBAC: Authorizing `identity:get_user(user_id=1eaed6d6b879446fb4ac79eba21dad47)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:43.893955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345602c6-5f01-4169-aa39-0a039aa218a2 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:43.895525 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 565/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:39:43 2022] GET /identity/v3/users/1eaed6d6b879446fb4ac79eba21dad47 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:43.901769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930cbb1-7706-44e2-9ff8-acb45da2c6bd None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:43.998435 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930cbb1-7706-44e2-9ff8-acb45da2c6bd None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:44.031785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a850423-997f-42d6-8179-e688a5e6b333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:44.032393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a850423-997f-42d6-8179-e688a5e6b333 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:44.032671 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a850423-997f-42d6-8179-e688a5e6b333 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:44.032943 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a850423-997f-42d6-8179-e688a5e6b333 None admin] PATH_INFO: `/v3/projects/4423f9c79d5048afbd9b0c6eae0bdde1` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:44.034037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a850423-997f-42d6-8179-e688a5e6b333 None admin] RBAC: Authorizing `identity:get_project(project_id=4423f9c79d5048afbd9b0c6eae0bdde1)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:44.035474 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a850423-997f-42d6-8179-e688a5e6b333 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:44.037064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 557/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:39:43 2022] GET /identity/v3/projects/4423f9c79d5048afbd9b0c6eae0bdde1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:44.042979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345602c6-5f01-4169-aa39-0a039aa218a2 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:44.136923 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345602c6-5f01-4169-aa39-0a039aa218a2 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:44.169360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b1dc9-ad96-4419-8e3e-481125df5da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:44.169934 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b1dc9-ad96-4419-8e3e-481125df5da8 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:44.170208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b1dc9-ad96-4419-8e3e-481125df5da8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:44.170475 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b1dc9-ad96-4419-8e3e-481125df5da8 None admin] PATH_INFO: `/v3/projects/4423f9c79d5048afbd9b0c6eae0bdde1/users/1eaed6d6b879446fb4ac79eba21dad47/roles/6fd5089acea448219d4c0a9a9424a3af` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:44.172603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b1dc9-ad96-4419-8e3e-481125df5da8 None admin] RBAC: Authorizing `identity:create_grant(project_id=4423f9c79d5048afbd9b0c6eae0bdde1, user_id=1eaed6d6b879446fb4ac79eba21dad47, role_id=6fd5089acea448219d4c0a9a9424a3af)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:44.174011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b1dc9-ad96-4419-8e3e-481125df5da8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:44.240269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 566/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:39:44 2022] PUT /identity/v3/projects/4423f9c79d5048afbd9b0c6eae0bdde1/users/1eaed6d6b879446fb4ac79eba21dad47/roles/6fd5089acea448219d4c0a9a9424a3af => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:39:46.087127 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee90d47-266c-4427-b0c6-56fc4571d7ff None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:46.087712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee90d47-266c-4427-b0c6-56fc4571d7ff None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:46.087810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee90d47-266c-4427-b0c6-56fc4571d7ff None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:46.089142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 558/1124] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:46.094297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8087b4e-259e-4ff1-a76a-8743acf68095 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:46.094712 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8087b4e-259e-4ff1-a76a-8743acf68095 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:46.094838 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8087b4e-259e-4ff1-a76a-8743acf68095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:46.112691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b8087b4e-259e-4ff1-a76a-8743acf68095 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:46.113858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b8087b4e-259e-4ff1-a76a-8743acf68095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:46.139319 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:46.178093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 567/1125] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:46.446941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee90d47-266c-4427-b0c6-56fc4571d7ff None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:46.462467 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:46.545511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee90d47-266c-4427-b0c6-56fc4571d7ff None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:46.577858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e73f9-fcc2-4655-9693-573867dcd450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:46.578490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e73f9-fcc2-4655-9693-573867dcd450 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:46.578767 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e73f9-fcc2-4655-9693-573867dcd450 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:46.579035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e73f9-fcc2-4655-9693-573867dcd450 None admin] PATH_INFO: `/v3/roles/6fd5089acea448219d4c0a9a9424a3af` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:46.580425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e73f9-fcc2-4655-9693-573867dcd450 None admin] RBAC: Authorizing `identity:get_role(role_id=6fd5089acea448219d4c0a9a9424a3af)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:46.581775 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e73f9-fcc2-4655-9693-573867dcd450 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:46.583045 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 559/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:39:46 2022] GET /identity/v3/roles/6fd5089acea448219d4c0a9a9424a3af => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:46.588776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8087b4e-259e-4ff1-a76a-8743acf68095 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:46.684666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8087b4e-259e-4ff1-a76a-8743acf68095 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:46.717203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718341a-abfd-4992-9f3a-6e24876e8e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:46.717753 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718341a-abfd-4992-9f3a-6e24876e8e8d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:46.718030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718341a-abfd-4992-9f3a-6e24876e8e8d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:46.718301 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718341a-abfd-4992-9f3a-6e24876e8e8d None admin] PATH_INFO: `/v3/users/1eaed6d6b879446fb4ac79eba21dad47` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:46.719439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718341a-abfd-4992-9f3a-6e24876e8e8d None admin] RBAC: Authorizing `identity:get_user(user_id=1eaed6d6b879446fb4ac79eba21dad47)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:46.720862 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718341a-abfd-4992-9f3a-6e24876e8e8d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:46.722408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 568/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:39:46 2022] GET /identity/v3/users/1eaed6d6b879446fb4ac79eba21dad47 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:46.728436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e73f9-fcc2-4655-9693-573867dcd450 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:46.808049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:46.825244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e73f9-fcc2-4655-9693-573867dcd450 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:46.858834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfa3e1-2fad-4f11-82d2-ebbe7327e145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:46.859472 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfa3e1-2fad-4f11-82d2-ebbe7327e145 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:46.859754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfa3e1-2fad-4f11-82d2-ebbe7327e145 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:46.860026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfa3e1-2fad-4f11-82d2-ebbe7327e145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:46.925085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfa3e1-2fad-4f11-82d2-ebbe7327e145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:46.926509 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfa3e1-2fad-4f11-82d2-ebbe7327e145 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:47.003119 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:47.021676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 560/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:39:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 14:39:47.028402 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718341a-abfd-4992-9f3a-6e24876e8e8d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:47.057507 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:47.124041 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718341a-abfd-4992-9f3a-6e24876e8e8d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:47.157692 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93016e01-80f5-411e-9b41-da30d59d9549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:47.158304 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93016e01-80f5-411e-9b41-da30d59d9549 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:47.158585 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93016e01-80f5-411e-9b41-da30d59d9549 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:47.158860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93016e01-80f5-411e-9b41-da30d59d9549 None admin] PATH_INFO: `/v3/projects/4423f9c79d5048afbd9b0c6eae0bdde1` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:47.159998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93016e01-80f5-411e-9b41-da30d59d9549 None admin] RBAC: Authorizing `identity:get_project(project_id=4423f9c79d5048afbd9b0c6eae0bdde1)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:47.161423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93016e01-80f5-411e-9b41-da30d59d9549 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:47.163036 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 569/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:39:47 2022] GET /identity/v3/projects/4423f9c79d5048afbd9b0c6eae0bdde1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:47.169181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfa3e1-2fad-4f11-82d2-ebbe7327e145 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:47.264276 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfa3e1-2fad-4f11-82d2-ebbe7327e145 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:47.297620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631539e-d2a8-4875-b6b6-efeb80e088e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:47.298211 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631539e-d2a8-4875-b6b6-efeb80e088e7 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:47.298488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631539e-d2a8-4875-b6b6-efeb80e088e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:47.298758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631539e-d2a8-4875-b6b6-efeb80e088e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:47.299501 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631539e-d2a8-4875-b6b6-efeb80e088e7 None admin] RBAC: Adding query filter params (role.id=6fd5089acea448219d4c0a9a9424a3af, scope.project.id=4423f9c79d5048afbd9b0c6eae0bdde1, user.id=1eaed6d6b879446fb4ac79eba21dad47) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:47.299811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631539e-d2a8-4875-b6b6-efeb80e088e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:47.301228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631539e-d2a8-4875-b6b6-efeb80e088e7 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:47.306790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 561/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:39:47 2022] GET /identity/v3/role_assignments?user.id=1eaed6d6b879446fb4ac79eba21dad47&scope.project.id=4423f9c79d5048afbd9b0c6eae0bdde1&role.id=6fd5089acea448219d4c0a9a9424a3af => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:39:49.225276 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a505daf-879c-4576-9c0f-258a7b721513 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:49.225855 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a505daf-879c-4576-9c0f-258a7b721513 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:49.225855 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a505daf-879c-4576-9c0f-258a7b721513 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:49.227284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 570/1131] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:49.232353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c82a3-2c0a-4164-ad14-f8b0a47faddf None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:49.232776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c82a3-2c0a-4164-ad14-f8b0a47faddf None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:49.232909 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c82a3-2c0a-4164-ad14-f8b0a47faddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:49.251212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-7b4c82a3-2c0a-4164-ad14-f8b0a47faddf None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:49.252393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7b4c82a3-2c0a-4164-ad14-f8b0a47faddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:49.294480 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:49.317972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 562/1132] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:49.591591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a505daf-879c-4576-9c0f-258a7b721513 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:49.685006 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:49.689502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a505daf-879c-4576-9c0f-258a7b721513 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:49.722772 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf53e5-8abd-461a-a05e-afb9e8f052bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:49.723450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf53e5-8abd-461a-a05e-afb9e8f052bf None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:49.723729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf53e5-8abd-461a-a05e-afb9e8f052bf None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:49.723999 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf53e5-8abd-461a-a05e-afb9e8f052bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:49.725107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf53e5-8abd-461a-a05e-afb9e8f052bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:49.727157 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf53e5-8abd-461a-a05e-afb9e8f052bf None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:49.728698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 571/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:49.734322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c82a3-2c0a-4164-ad14-f8b0a47faddf None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:49.829292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c82a3-2c0a-4164-ad14-f8b0a47faddf None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:49.862724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de2943-f9b3-4457-a263-f3e4a250b91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:49.863294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de2943-f9b3-4457-a263-f3e4a250b91a None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:49.863626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de2943-f9b3-4457-a263-f3e4a250b91a None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:49.863881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de2943-f9b3-4457-a263-f3e4a250b91a None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:49.864542 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de2943-f9b3-4457-a263-f3e4a250b91a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:49.865939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de2943-f9b3-4457-a263-f3e4a250b91a None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:49.950369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 563/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 30 14:39:49 2022] POST /identity/v3/projects => generated 311 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:39:51.853951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259bd34-29bb-4e5a-86df-67c4d09d1008 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:51.854473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259bd34-29bb-4e5a-86df-67c4d09d1008 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:51.854473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259bd34-29bb-4e5a-86df-67c4d09d1008 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:51.855860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 572/1135] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:51.861009 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e76a0-2859-4258-a5e8-2ab86cab2577 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:51.861423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e76a0-2859-4258-a5e8-2ab86cab2577 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:51.861558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e76a0-2859-4258-a5e8-2ab86cab2577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:51.880140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-f16e76a0-2859-4258-a5e8-2ab86cab2577 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:51.881320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-f16e76a0-2859-4258-a5e8-2ab86cab2577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:51.904065 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:51.945441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 564/1136] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:52.214282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259bd34-29bb-4e5a-86df-67c4d09d1008 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:52.313947 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259bd34-29bb-4e5a-86df-67c4d09d1008 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:52.345949 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b969b71-72b7-4035-b9aa-ced61c445269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:52.346553 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b969b71-72b7-4035-b9aa-ced61c445269 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:52.346826 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b969b71-72b7-4035-b9aa-ced61c445269 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:52.347088 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b969b71-72b7-4035-b9aa-ced61c445269 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:52.348239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b969b71-72b7-4035-b9aa-ced61c445269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:52.349646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b969b71-72b7-4035-b9aa-ced61c445269 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:52.351730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 573/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:39:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:52.357460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e76a0-2859-4258-a5e8-2ab86cab2577 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:52.452821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e76a0-2859-4258-a5e8-2ab86cab2577 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:52.486837 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c409fa0-d751-4228-972b-ea72dec13903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:52.487454 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c409fa0-d751-4228-972b-ea72dec13903 None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:52.487731 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c409fa0-d751-4228-972b-ea72dec13903 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:52.487998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c409fa0-d751-4228-972b-ea72dec13903 None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:52.488646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c409fa0-d751-4228-972b-ea72dec13903 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:52.490056 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c409fa0-d751-4228-972b-ea72dec13903 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:52.565947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 565/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:39:52 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:39:54.492714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec990f5-2ebd-4f09-bd61-4ba1090baca4 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:54.493242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec990f5-2ebd-4f09-bd61-4ba1090baca4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:54.493242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec990f5-2ebd-4f09-bd61-4ba1090baca4 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:54.494581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 574/1139] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:54.499718 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3202cd8-7961-4c9d-a1af-f96e7fc9a169 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:54.500153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3202cd8-7961-4c9d-a1af-f96e7fc9a169 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:54.500247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3202cd8-7961-4c9d-a1af-f96e7fc9a169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:54.518544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-b3202cd8-7961-4c9d-a1af-f96e7fc9a169 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:54.519699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b3202cd8-7961-4c9d-a1af-f96e7fc9a169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:54.528959 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:54.585482 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 566/1140] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:54.853087 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec990f5-2ebd-4f09-bd61-4ba1090baca4 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:54.887204 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:54.951390 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec990f5-2ebd-4f09-bd61-4ba1090baca4 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:54.984507 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15effac1-1609-406b-9ddc-25996cd096f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:54.985117 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15effac1-1609-406b-9ddc-25996cd096f6 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:54.985396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15effac1-1609-406b-9ddc-25996cd096f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:54.985669 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15effac1-1609-406b-9ddc-25996cd096f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:54.995507 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15effac1-1609-406b-9ddc-25996cd096f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:39:55.000306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-15effac1-1609-406b-9ddc-25996cd096f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:39:55.000306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:55.000306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:55.000306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:55.000306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:55.000306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:55.000306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:55.000306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:55.000306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:55.000306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:55.000306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:55.000306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:55.000306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:39:55.000306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:39:55.000306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:39:55.000306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:39:55.001450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:39:55.001450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:39:55.001450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:39:55.001450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:39:55.001450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:55.001450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:55.001450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:55.001450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:55.001450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:55.001450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:55.001450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:55.001450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:55.001450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:55.001450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:55.001450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:55.001450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:55.002787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:55.002787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:55.002787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:55.002787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:55.002787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:39:55.002787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:39:55.002787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:39:55.002787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:39:55.002787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:39:55.002787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:39:55.002787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:39:55.002787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:55.002787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 575/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:39:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:55.007246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3202cd8-7961-4c9d-a1af-f96e7fc9a169 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:55.102395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3202cd8-7961-4c9d-a1af-f96e7fc9a169 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:55.134415 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4cbb4-ea66-4e6a-8fdd-1496cd6cc8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:55.134942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4cbb4-ea66-4e6a-8fdd-1496cd6cc8aa None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:55.135247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4cbb4-ea66-4e6a-8fdd-1496cd6cc8aa None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:55.135518 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4cbb4-ea66-4e6a-8fdd-1496cd6cc8aa None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:55.136173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4cbb4-ea66-4e6a-8fdd-1496cd6cc8aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:55.136471 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4cbb4-ea66-4e6a-8fdd-1496cd6cc8aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:55.137812 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4cbb4-ea66-4e6a-8fdd-1496cd6cc8aa None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:55.145507 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 567/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:39:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:55.151799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15effac1-1609-406b-9ddc-25996cd096f6 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:55.242114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:55.248628 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15effac1-1609-406b-9ddc-25996cd096f6 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:55.282998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c173c-ed95-4494-a828-1da81fa837e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:55.283646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c173c-ed95-4494-a828-1da81fa837e4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:55.283926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c173c-ed95-4494-a828-1da81fa837e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:55.284200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c173c-ed95-4494-a828-1da81fa837e4 None admin] PATH_INFO: `/v3/users/fae1d12166774bf5a36bce067b2d5aea` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:55.296993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c173c-ed95-4494-a828-1da81fa837e4 None admin] RBAC: Authorizing `identity:get_user(user_id=fae1d12166774bf5a36bce067b2d5aea)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:55.298431 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c173c-ed95-4494-a828-1da81fa837e4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:55.300066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 576/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:39:55 2022] GET /identity/v3/users/fae1d12166774bf5a36bce067b2d5aea => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:55.306053 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4cbb4-ea66-4e6a-8fdd-1496cd6cc8aa None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:55.315291 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:55.403950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4cbb4-ea66-4e6a-8fdd-1496cd6cc8aa None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:55.436007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00284cad-6159-48dd-8b9a-c53a498584d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:55.436601 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284cad-6159-48dd-8b9a-c53a498584d5 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:55.436880 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284cad-6159-48dd-8b9a-c53a498584d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:55.437150 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284cad-6159-48dd-8b9a-c53a498584d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:55.502952 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00284cad-6159-48dd-8b9a-c53a498584d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:55.504423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00284cad-6159-48dd-8b9a-c53a498584d5 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:55.512606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:55.600899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 568/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:39:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:39:55.606883 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c173c-ed95-4494-a828-1da81fa837e4 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:55.703590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c173c-ed95-4494-a828-1da81fa837e4 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:55.735688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc80657-db61-4fa2-a9a9-4b77726a4a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:55.736297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc80657-db61-4fa2-a9a9-4b77726a4a1a None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:55.736578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc80657-db61-4fa2-a9a9-4b77726a4a1a None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:55.736848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc80657-db61-4fa2-a9a9-4b77726a4a1a None admin] PATH_INFO: `/v3/projects/aa609e9ecef54d399d25d8252fbd8484` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:55.737956 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc80657-db61-4fa2-a9a9-4b77726a4a1a None admin] RBAC: Authorizing `identity:get_project(project_id=aa609e9ecef54d399d25d8252fbd8484)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:55.739418 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc80657-db61-4fa2-a9a9-4b77726a4a1a None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:55.741547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 577/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:39:55 2022] GET /identity/v3/projects/aa609e9ecef54d399d25d8252fbd8484 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:55.747527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00284cad-6159-48dd-8b9a-c53a498584d5 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:55.843570 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00284cad-6159-48dd-8b9a-c53a498584d5 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:55.850650 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:55.876263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c194916-c8f3-4a8b-9a40-8bacb990207d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:55.876851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c194916-c8f3-4a8b-9a40-8bacb990207d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:55.877125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c194916-c8f3-4a8b-9a40-8bacb990207d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:55.877392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c194916-c8f3-4a8b-9a40-8bacb990207d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:55.878104 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c194916-c8f3-4a8b-9a40-8bacb990207d None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=aa609e9ecef54d399d25d8252fbd8484, user.id=fae1d12166774bf5a36bce067b2d5aea) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:55.878406 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c194916-c8f3-4a8b-9a40-8bacb990207d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:55.879839 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c194916-c8f3-4a8b-9a40-8bacb990207d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:55.885615 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 569/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:39:55 2022] GET /identity/v3/role_assignments?user.id=fae1d12166774bf5a36bce067b2d5aea&scope.project.id=aa609e9ecef54d399d25d8252fbd8484&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:57.758296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124d50f-5015-4bc7-ae7a-64d88275dd5e None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:57.758825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124d50f-5015-4bc7-ae7a-64d88275dd5e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:57.758825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124d50f-5015-4bc7-ae7a-64d88275dd5e None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:57.760245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 578/1147] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:39:57.765402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52efe9-4215-4b0d-8d6a-ebd94d73085d None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:57.765828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52efe9-4215-4b0d-8d6a-ebd94d73085d None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:57.765950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52efe9-4215-4b0d-8d6a-ebd94d73085d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:57.784830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-8e52efe9-4215-4b0d-8d6a-ebd94d73085d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:39:57.785995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8e52efe9-4215-4b0d-8d6a-ebd94d73085d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:39:57.849455 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 570/1148] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:39:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:39:58.121856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124d50f-5015-4bc7-ae7a-64d88275dd5e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:58.142949 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:58.220042 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124d50f-5015-4bc7-ae7a-64d88275dd5e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:58.253767 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d2da4-929b-4c04-8c57-9cb23ce8b848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:58.254377 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d2da4-929b-4c04-8c57-9cb23ce8b848 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:58.254652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d2da4-929b-4c04-8c57-9cb23ce8b848 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:58.254918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d2da4-929b-4c04-8c57-9cb23ce8b848 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:58.265139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d2da4-929b-4c04-8c57-9cb23ce8b848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:39:58.269824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-c32d2da4-929b-4c04-8c57-9cb23ce8b848 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:39:58.269824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:39:58.269824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:39:58.269824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:39:58.269824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:39:58.269824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:39:58.269824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:39:58.269824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:39:58.269824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:39:58.269824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:39:58.269824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:39:58.269824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:39:58.269824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:39:58.269824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:39:58.269824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:39:58.269824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:39:58.270944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:39:58.270944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:39:58.270944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:39:58.270944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:39:58.270944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:39:58.270944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:39:58.270944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:39:58.270944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:39:58.270944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:39:58.270944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:39:58.270944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:39:58.270944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:39:58.270944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:39:58.270944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:39:58.270944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:39:58.270944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:39:58.273022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:39:58.273022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:39:58.273022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:39:58.273022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:39:58.273022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:39:58.273022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:39:58.273022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:39:58.273022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:39:58.273022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:39:58.273022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:39:58.273022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:39:58.273022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:39:58.273022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 579/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:39:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:39:58.277747 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52efe9-4215-4b0d-8d6a-ebd94d73085d None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:58.370293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:58.373126 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52efe9-4215-4b0d-8d6a-ebd94d73085d None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:58.407500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2805f4-f05a-489e-801f-48379284d2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:58.408087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2805f4-f05a-489e-801f-48379284d2a2 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:58.408359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2805f4-f05a-489e-801f-48379284d2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:58.408622 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2805f4-f05a-489e-801f-48379284d2a2 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:58.409276 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2805f4-f05a-489e-801f-48379284d2a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:39:58.409569 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2805f4-f05a-489e-801f-48379284d2a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:58.410919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2805f4-f05a-489e-801f-48379284d2a2 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:58.418881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 571/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:39:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:58.424780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d2da4-929b-4c04-8c57-9cb23ce8b848 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:58.502875 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:58.521384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d2da4-929b-4c04-8c57-9cb23ce8b848 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:58.554666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f25b4a-de32-4b08-b4e6-5a64764a4f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:58.555437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f25b4a-de32-4b08-b4e6-5a64764a4f97 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:58.555721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f25b4a-de32-4b08-b4e6-5a64764a4f97 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:58.555993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f25b4a-de32-4b08-b4e6-5a64764a4f97 None admin] PATH_INFO: `/v3/users/fae1d12166774bf5a36bce067b2d5aea` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:58.557121 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f25b4a-de32-4b08-b4e6-5a64764a4f97 None admin] RBAC: Authorizing `identity:get_user(user_id=fae1d12166774bf5a36bce067b2d5aea)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:58.558560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f25b4a-de32-4b08-b4e6-5a64764a4f97 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:58.560168 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 580/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:39:58 2022] GET /identity/v3/users/fae1d12166774bf5a36bce067b2d5aea => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:58.565919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2805f4-f05a-489e-801f-48379284d2a2 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:58.661691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2805f4-f05a-489e-801f-48379284d2a2 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:58.695924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6c0c7-779d-43ee-b5d4-e20ce933a2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:58.696524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6c0c7-779d-43ee-b5d4-e20ce933a2ab None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:58.696797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6c0c7-779d-43ee-b5d4-e20ce933a2ab None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:58.697063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6c0c7-779d-43ee-b5d4-e20ce933a2ab None admin] PATH_INFO: `/v3/projects/aa609e9ecef54d399d25d8252fbd8484` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:58.698154 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6c0c7-779d-43ee-b5d4-e20ce933a2ab None admin] RBAC: Authorizing `identity:get_project(project_id=aa609e9ecef54d399d25d8252fbd8484)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:58.699602 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6c0c7-779d-43ee-b5d4-e20ce933a2ab None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:58.701197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 572/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:39:58 2022] GET /identity/v3/projects/aa609e9ecef54d399d25d8252fbd8484 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:39:58.707650 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f25b4a-de32-4b08-b4e6-5a64764a4f97 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:39:58.803926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f25b4a-de32-4b08-b4e6-5a64764a4f97 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:39:58.836880 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d84a65-12cb-40c5-b59f-dc4cbe28d6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:39:58.837523 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d84a65-12cb-40c5-b59f-dc4cbe28d6b4 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:39:58.837802 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d84a65-12cb-40c5-b59f-dc4cbe28d6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:39:58.838075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d84a65-12cb-40c5-b59f-dc4cbe28d6b4 None admin] PATH_INFO: `/v3/projects/aa609e9ecef54d399d25d8252fbd8484/users/fae1d12166774bf5a36bce067b2d5aea/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:39:58.839992 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d84a65-12cb-40c5-b59f-dc4cbe28d6b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa609e9ecef54d399d25d8252fbd8484, user_id=fae1d12166774bf5a36bce067b2d5aea, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:39:58.841411 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d84a65-12cb-40c5-b59f-dc4cbe28d6b4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:39:58.949066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 581/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:39:58 2022] PUT /identity/v3/projects/aa609e9ecef54d399d25d8252fbd8484/users/fae1d12166774bf5a36bce067b2d5aea/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:40:00.820447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04063ba-44f3-4503-a4f0-2850c48e6a45 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:00.821030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04063ba-44f3-4503-a4f0-2850c48e6a45 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:00.821030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04063ba-44f3-4503-a4f0-2850c48e6a45 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:00.822313 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 573/1154] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:00.827624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437f0c4-9967-47af-9d49-6a459ec60e34 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:00.828035 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437f0c4-9967-47af-9d49-6a459ec60e34 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:00.828174 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437f0c4-9967-47af-9d49-6a459ec60e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:00.845990 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0437f0c4-9967-47af-9d49-6a459ec60e34 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:00.847213 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0437f0c4-9967-47af-9d49-6a459ec60e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:00.849137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:00.912733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 582/1155] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:40:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:40:01.180892 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04063ba-44f3-4503-a4f0-2850c48e6a45 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:01.211113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:01.282235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04063ba-44f3-4503-a4f0-2850c48e6a45 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:01.315178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc7ec9-24cf-4609-ad34-754c198406bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:01.315799 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc7ec9-24cf-4609-ad34-754c198406bd None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:01.316077 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc7ec9-24cf-4609-ad34-754c198406bd None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:01.316347 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc7ec9-24cf-4609-ad34-754c198406bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:01.326788 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc7ec9-24cf-4609-ad34-754c198406bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:40:01.331634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-51dc7ec9-24cf-4609-ad34-754c198406bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:40:01.331634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:40:01.331634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:40:01.331634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:40:01.331634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:40:01.331634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:40:01.331634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:40:01.331634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:40:01.331634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:40:01.331634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:40:01.331634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:40:01.331634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:40:01.331634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:40:01.331634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:40:01.331634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:40:01.331634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:40:01.332728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:40:01.332728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:40:01.332728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:40:01.332728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:40:01.332728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:40:01.332728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:40:01.332728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:40:01.332728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:40:01.332728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:40:01.332728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:40:01.332728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:40:01.332728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:40:01.332728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:40:01.332728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:40:01.332728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:40:01.332728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:40:01.333640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:40:01.333640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:40:01.333640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:40:01.333640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:40:01.333640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:40:01.333640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:40:01.333640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:40:01.333640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:40:01.333640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:40:01.333640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:40:01.333640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:40:01.333640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:40:01.333640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 574/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:40:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:40:01.339296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437f0c4-9967-47af-9d49-6a459ec60e34 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:01.434102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437f0c4-9967-47af-9d49-6a459ec60e34 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:01.467383 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fff466-a73c-41ec-b3ef-ee812ffc6c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:01.467969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fff466-a73c-41ec-b3ef-ee812ffc6c16 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:01.468242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fff466-a73c-41ec-b3ef-ee812ffc6c16 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:01.468508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fff466-a73c-41ec-b3ef-ee812ffc6c16 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:01.469157 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fff466-a73c-41ec-b3ef-ee812ffc6c16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:40:01.469493 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fff466-a73c-41ec-b3ef-ee812ffc6c16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:01.470856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fff466-a73c-41ec-b3ef-ee812ffc6c16 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:01.475727 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:01.478954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 583/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:40:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:01.484841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc7ec9-24cf-4609-ad34-754c198406bd None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:01.572114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:01.580921 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc7ec9-24cf-4609-ad34-754c198406bd None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:01.613611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3e626-53a4-4dff-a74c-6f2b3caee7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:01.614220 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3e626-53a4-4dff-a74c-6f2b3caee7e3 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:01.614497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3e626-53a4-4dff-a74c-6f2b3caee7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:01.614766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3e626-53a4-4dff-a74c-6f2b3caee7e3 None admin] PATH_INFO: `/v3/users/fae1d12166774bf5a36bce067b2d5aea` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:01.615934 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3e626-53a4-4dff-a74c-6f2b3caee7e3 None admin] RBAC: Authorizing `identity:get_user(user_id=fae1d12166774bf5a36bce067b2d5aea)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:01.617346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3e626-53a4-4dff-a74c-6f2b3caee7e3 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:01.618906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 575/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:40:01 2022] GET /identity/v3/users/fae1d12166774bf5a36bce067b2d5aea => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:01.625008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fff466-a73c-41ec-b3ef-ee812ffc6c16 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:01.721388 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fff466-a73c-41ec-b3ef-ee812ffc6c16 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:01.753285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468b60b-0b50-44df-960d-ced6e494afb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:01.753862 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468b60b-0b50-44df-960d-ced6e494afb1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:01.754137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468b60b-0b50-44df-960d-ced6e494afb1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:01.754405 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468b60b-0b50-44df-960d-ced6e494afb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:01.817763 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:01.820433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468b60b-0b50-44df-960d-ced6e494afb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:01.821847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468b60b-0b50-44df-960d-ced6e494afb1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:01.914032 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 584/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:40:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:40:01.919903 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3e626-53a4-4dff-a74c-6f2b3caee7e3 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:02.015863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3e626-53a4-4dff-a74c-6f2b3caee7e3 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:02.048688 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a5a08-02b2-4e7c-9ede-6c7d58ed905f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:02.049292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a5a08-02b2-4e7c-9ede-6c7d58ed905f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:02.049567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a5a08-02b2-4e7c-9ede-6c7d58ed905f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:02.049836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a5a08-02b2-4e7c-9ede-6c7d58ed905f None admin] PATH_INFO: `/v3/projects/aa609e9ecef54d399d25d8252fbd8484` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:02.050931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a5a08-02b2-4e7c-9ede-6c7d58ed905f None admin] RBAC: Authorizing `identity:get_project(project_id=aa609e9ecef54d399d25d8252fbd8484)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:02.052373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a5a08-02b2-4e7c-9ede-6c7d58ed905f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:02.054526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 576/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:40:01 2022] GET /identity/v3/projects/aa609e9ecef54d399d25d8252fbd8484 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:02.060445 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468b60b-0b50-44df-960d-ced6e494afb1 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:02.152843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:02.155673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468b60b-0b50-44df-960d-ced6e494afb1 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:02.188250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbf95a-143d-49b0-933e-e5fab137520d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:02.188833 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbf95a-143d-49b0-933e-e5fab137520d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:02.189110 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbf95a-143d-49b0-933e-e5fab137520d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:02.189380 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbf95a-143d-49b0-933e-e5fab137520d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:02.190097 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbf95a-143d-49b0-933e-e5fab137520d None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=aa609e9ecef54d399d25d8252fbd8484, user.id=fae1d12166774bf5a36bce067b2d5aea) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:40:02.190404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbf95a-143d-49b0-933e-e5fab137520d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:02.191843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbf95a-143d-49b0-933e-e5fab137520d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:02.197310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 585/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:40:02 2022] GET /identity/v3/role_assignments?user.id=fae1d12166774bf5a36bce067b2d5aea&scope.project.id=aa609e9ecef54d399d25d8252fbd8484&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:40:04.105164 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07538bbb-bef9-4241-a498-23f188ad0640 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:04.105689 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07538bbb-bef9-4241-a498-23f188ad0640 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:04.105689 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07538bbb-bef9-4241-a498-23f188ad0640 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:04.107027 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 577/1162] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:04.112287 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015d0c8-26ff-4778-a8fc-bd6ea85a4776 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:04.112716 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015d0c8-26ff-4778-a8fc-bd6ea85a4776 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:04.112840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015d0c8-26ff-4778-a8fc-bd6ea85a4776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:04.130553 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-7015d0c8-26ff-4778-a8fc-bd6ea85a4776 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:04.131771 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7015d0c8-26ff-4778-a8fc-bd6ea85a4776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:04.195846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 586/1163] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:40:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:40:04.466290 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07538bbb-bef9-4241-a498-23f188ad0640 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:04.493160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:04.566148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07538bbb-bef9-4241-a498-23f188ad0640 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:04.599779 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19087781-f90f-4c27-b568-65bc2f73470c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:04.600371 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19087781-f90f-4c27-b568-65bc2f73470c None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:04.600648 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19087781-f90f-4c27-b568-65bc2f73470c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:04.600919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19087781-f90f-4c27-b568-65bc2f73470c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:04.666043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19087781-f90f-4c27-b568-65bc2f73470c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:04.667524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19087781-f90f-4c27-b568-65bc2f73470c None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:04.692503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:04.761929 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 578/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:40:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:40:04.767880 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015d0c8-26ff-4778-a8fc-bd6ea85a4776 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:04.856809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:04.863146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015d0c8-26ff-4778-a8fc-bd6ea85a4776 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:04.896492 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758260e5-2372-4578-b8d7-4f4f22d27dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:04.897134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758260e5-2372-4578-b8d7-4f4f22d27dde None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:04.897414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758260e5-2372-4578-b8d7-4f4f22d27dde None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:04.897688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758260e5-2372-4578-b8d7-4f4f22d27dde None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:04.904087 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758260e5-2372-4578-b8d7-4f4f22d27dde None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:04.905517 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758260e5-2372-4578-b8d7-4f4f22d27dde None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-758260e5-2372-4578-b8d7-4f4f22d27dde None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:40:04.916501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:40:04.917688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:40:04.917688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:40:04.917688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:40:04.917688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:40:04.917688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:40:04.917688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:40:04.917688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:40:04.917688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:40:04.917688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:40:04.917688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:40:04.917688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:40:04.917688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:40:04.917688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:40:04.917688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:40:04.917688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:40:04.917688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:40:04.918730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:40:04.918730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:40:04.918730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:40:04.918730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:40:04.918730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:40:04.918730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:40:04.918730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:40:04.918730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:40:04.918730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:40:04.918730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:40:04.918730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 30 14:40:04.918730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:40:04.918730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 14:40:04.918730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:40:04.918730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 587/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 30 14:40:04 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:40:04.923397 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19087781-f90f-4c27-b568-65bc2f73470c None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:05.018958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19087781-f90f-4c27-b568-65bc2f73470c None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:05.043214 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:05.051257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245a15b-188b-4855-8d48-d3d8825fe7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:05.051841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245a15b-188b-4855-8d48-d3d8825fe7ae None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:05.056483 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245a15b-188b-4855-8d48-d3d8825fe7ae None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:05.056483 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245a15b-188b-4855-8d48-d3d8825fe7ae None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:05.056483 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245a15b-188b-4855-8d48-d3d8825fe7ae None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:40:05.056483 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245a15b-188b-4855-8d48-d3d8825fe7ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:05.056483 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245a15b-188b-4855-8d48-d3d8825fe7ae None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:05.061974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 579/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Aug 30 14:40:04 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:06.918726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72029c1b-09f7-4654-8067-967818a23e5d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:06.919269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72029c1b-09f7-4654-8067-967818a23e5d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:06.919351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72029c1b-09f7-4654-8067-967818a23e5d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:06.920643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 588/1167] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:06.925852 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea33ba5-93b4-4406-927f-10db220945c2 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:06.926241 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea33ba5-93b4-4406-927f-10db220945c2 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:06.926382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea33ba5-93b4-4406-927f-10db220945c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:06.945741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-2ea33ba5-93b4-4406-927f-10db220945c2 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:06.947242 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-2ea33ba5-93b4-4406-927f-10db220945c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:07.013807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 580/1168] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:40:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:40:07.282554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72029c1b-09f7-4654-8067-967818a23e5d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:07.381304 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72029c1b-09f7-4654-8067-967818a23e5d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:07.413437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e053b-f2b0-4678-add7-351d4610d0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:07.414023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e053b-f2b0-4678-add7-351d4610d0a9 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:07.414297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e053b-f2b0-4678-add7-351d4610d0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:07.414564 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e053b-f2b0-4678-add7-351d4610d0a9 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:07.415256 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e053b-f2b0-4678-add7-351d4610d0a9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:07.416602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e053b-f2b0-4678-add7-351d4610d0a9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:07.496124 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 589/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Tue Aug 30 14:40:07 2022] POST /identity/v3/domains => generated 267 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:40:09.383735 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34352330-49b4-42fd-9548-6338419624bf None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:09.384224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34352330-49b4-42fd-9548-6338419624bf None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:09.384337 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34352330-49b4-42fd-9548-6338419624bf None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:09.385600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 581/1170] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:09.390637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b4325-5bfa-4062-bb8e-bf90e5d6a319 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:09.390998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b4325-5bfa-4062-bb8e-bf90e5d6a319 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:09.391223 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b4325-5bfa-4062-bb8e-bf90e5d6a319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:09.410434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-377b4325-5bfa-4062-bb8e-bf90e5d6a319 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:09.411651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-377b4325-5bfa-4062-bb8e-bf90e5d6a319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:09.421185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:09.475477 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 590/1171] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:40:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:40:09.740661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34352330-49b4-42fd-9548-6338419624bf None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:09.837995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34352330-49b4-42fd-9548-6338419624bf None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:09.857478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:09.871224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f31ea1-eb6a-43b6-8cf3-17389d367c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:09.871830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f31ea1-eb6a-43b6-8cf3-17389d367c1b None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:09.872104 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f31ea1-eb6a-43b6-8cf3-17389d367c1b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:09.872370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f31ea1-eb6a-43b6-8cf3-17389d367c1b None admin] PATH_INFO: `/v3/domains/f20bacf0887c49569b135943171a6590` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:09.882254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f31ea1-eb6a-43b6-8cf3-17389d367c1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=f20bacf0887c49569b135943171a6590)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:09.883693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f31ea1-eb6a-43b6-8cf3-17389d367c1b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:09.885240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 582/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Tue Aug 30 14:40:09 2022] GET /identity/v3/domains/f20bacf0887c49569b135943171a6590 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:09.891249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b4325-5bfa-4062-bb8e-bf90e5d6a319 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:09.986458 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b4325-5bfa-4062-bb8e-bf90e5d6a319 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:10.018436 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c6bc1-9168-44c1-a086-06cf7b279779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:10.018970 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c6bc1-9168-44c1-a086-06cf7b279779 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:10.019275 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c6bc1-9168-44c1-a086-06cf7b279779 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:10.019548 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c6bc1-9168-44c1-a086-06cf7b279779 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:10.020199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c6bc1-9168-44c1-a086-06cf7b279779 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:10.022390 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c6bc1-9168-44c1-a086-06cf7b279779 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:10.036723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:10.118549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 591/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 30 14:40:09 2022] POST /identity/v3/projects => generated 361 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:40:12.006332 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9db763-27d7-49f3-a85b-ac4438e5f5ee None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:12.006882 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9db763-27d7-49f3-a85b-ac4438e5f5ee None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:12.006882 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9db763-27d7-49f3-a85b-ac4438e5f5ee None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:12.008238 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 583/1174] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:12.013408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27bb6e-5a34-4a23-97a7-8e8e6a9d48bd None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:12.013796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27bb6e-5a34-4a23-97a7-8e8e6a9d48bd None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:12.013953 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27bb6e-5a34-4a23-97a7-8e8e6a9d48bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:12.031914 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-df27bb6e-5a34-4a23-97a7-8e8e6a9d48bd None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:12.033085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-df27bb6e-5a34-4a23-97a7-8e8e6a9d48bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:12.096681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 592/1175] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:40:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:40:12.378070 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9db763-27d7-49f3-a85b-ac4438e5f5ee None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:12.478097 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9db763-27d7-49f3-a85b-ac4438e5f5ee None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:12.510085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762045d4-32c3-490b-886a-e643d3f30a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:12.510691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762045d4-32c3-490b-886a-e643d3f30a59 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:12.510964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762045d4-32c3-490b-886a-e643d3f30a59 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:12.511258 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762045d4-32c3-490b-886a-e643d3f30a59 None admin] PATH_INFO: `/v3/domains/f20bacf0887c49569b135943171a6590` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:12.512343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762045d4-32c3-490b-886a-e643d3f30a59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f20bacf0887c49569b135943171a6590)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:12.513757 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762045d4-32c3-490b-886a-e643d3f30a59 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:12.515306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 584/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Tue Aug 30 14:40:12 2022] GET /identity/v3/domains/f20bacf0887c49569b135943171a6590 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:12.522014 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27bb6e-5a34-4a23-97a7-8e8e6a9d48bd None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:12.615898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27bb6e-5a34-4a23-97a7-8e8e6a9d48bd None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:12.640420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:12.649527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2309f5f-e127-4839-9099-3bc4dd4cfa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:12.650120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2309f5f-e127-4839-9099-3bc4dd4cfa34 None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:12.650396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2309f5f-e127-4839-9099-3bc4dd4cfa34 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:12.650664 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2309f5f-e127-4839-9099-3bc4dd4cfa34 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:12.651345 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2309f5f-e127-4839-9099-3bc4dd4cfa34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:12.652769 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2309f5f-e127-4839-9099-3bc4dd4cfa34 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:12.762985 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 593/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:40:12 2022] POST /identity/v3/users => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:40:14.678204 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288808e-af93-481b-947f-f2025cbe5f23 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:14.678705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288808e-af93-481b-947f-f2025cbe5f23 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:14.678705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288808e-af93-481b-947f-f2025cbe5f23 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:14.680017 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 585/1178] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:14.685190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4fb65-de49-416b-a6a1-0d87cd124d9d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:14.685579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4fb65-de49-416b-a6a1-0d87cd124d9d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:14.685731 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4fb65-de49-416b-a6a1-0d87cd124d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:14.703318 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0cc4fb65-de49-416b-a6a1-0d87cd124d9d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:14.704481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0cc4fb65-de49-416b-a6a1-0d87cd124d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:14.767694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 594/1179] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:40:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:40:15.037791 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288808e-af93-481b-947f-f2025cbe5f23 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:15.086892 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:15.137002 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288808e-af93-481b-947f-f2025cbe5f23 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:15.169431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df0ded-d589-4ab1-adfc-27dc23f8f7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:15.170036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df0ded-d589-4ab1-adfc-27dc23f8f7d5 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:15.170320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df0ded-d589-4ab1-adfc-27dc23f8f7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:15.170590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df0ded-d589-4ab1-adfc-27dc23f8f7d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:15.181281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df0ded-d589-4ab1-adfc-27dc23f8f7d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:40:15.185967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-66df0ded-d589-4ab1-adfc-27dc23f8f7d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:40:15.185967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:40:15.185967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:40:15.185967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:40:15.185967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:40:15.185967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:40:15.185967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:40:15.185967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:40:15.185967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:40:15.185967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:40:15.185967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:40:15.185967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:40:15.185967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:40:15.185967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:40:15.185967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:40:15.185967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:40:15.187035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:40:15.187035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:40:15.187035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:40:15.187035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:40:15.187035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:40:15.187035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:40:15.187035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:40:15.187035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:40:15.187035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:40:15.187035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:40:15.187035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:40:15.187035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:40:15.187035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:40:15.187035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:40:15.187035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:40:15.187035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:40:15.187947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:40:15.187947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:40:15.187947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:40:15.187947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:40:15.187947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:40:15.187947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:40:15.187947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:40:15.187947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:40:15.187947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:40:15.187947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:40:15.187947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:40:15.187947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:40:15.187947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 586/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:40:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:40:15.193673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4fb65-de49-416b-a6a1-0d87cd124d9d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:15.293310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4fb65-de49-416b-a6a1-0d87cd124d9d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:15.306495 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:15.327759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0348300e-4f56-4d42-94ff-c584fb3efe84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:15.328358 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348300e-4f56-4d42-94ff-c584fb3efe84 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:15.328638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348300e-4f56-4d42-94ff-c584fb3efe84 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:15.328910 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348300e-4f56-4d42-94ff-c584fb3efe84 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:15.329569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0348300e-4f56-4d42-94ff-c584fb3efe84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:40:15.329872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0348300e-4f56-4d42-94ff-c584fb3efe84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:15.331275 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0348300e-4f56-4d42-94ff-c584fb3efe84 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:15.338969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 595/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:40:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:15.344810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df0ded-d589-4ab1-adfc-27dc23f8f7d5 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:15.441861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df0ded-d589-4ab1-adfc-27dc23f8f7d5 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:15.451614 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:15.475900 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3509db-d15e-4bdd-b752-9c0827de172c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:15.497072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3509db-d15e-4bdd-b752-9c0827de172c None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:15.497072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3509db-d15e-4bdd-b752-9c0827de172c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:15.497072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3509db-d15e-4bdd-b752-9c0827de172c None admin] PATH_INFO: `/v3/users/7f9da35db15a4b1398eb28c741c43764` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:15.497072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3509db-d15e-4bdd-b752-9c0827de172c None admin] RBAC: Authorizing `identity:get_user(user_id=7f9da35db15a4b1398eb28c741c43764)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:15.497072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3509db-d15e-4bdd-b752-9c0827de172c None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:15.497072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 587/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:40:15 2022] GET /identity/v3/users/7f9da35db15a4b1398eb28c741c43764 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:15.499630 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0348300e-4f56-4d42-94ff-c584fb3efe84 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:15.597929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0348300e-4f56-4d42-94ff-c584fb3efe84 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:15.630327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfc0b7-9735-40c1-8ca1-b0a7aedd11b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:15.630915 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfc0b7-9735-40c1-8ca1-b0a7aedd11b5 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:15.631218 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfc0b7-9735-40c1-8ca1-b0a7aedd11b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:15.631488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfc0b7-9735-40c1-8ca1-b0a7aedd11b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:15.671384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:15.695017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cfc0b7-9735-40c1-8ca1-b0a7aedd11b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:15.696466 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cfc0b7-9735-40c1-8ca1-b0a7aedd11b5 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:15.790825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 596/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:40:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:40:15.796761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3509db-d15e-4bdd-b752-9c0827de172c None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:15.892427 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3509db-d15e-4bdd-b752-9c0827de172c None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:15.925466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f31c91-1906-4c45-8f7a-0d3aa3b65017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:15.926075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f31c91-1906-4c45-8f7a-0d3aa3b65017 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:15.926346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f31c91-1906-4c45-8f7a-0d3aa3b65017 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:15.926611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f31c91-1906-4c45-8f7a-0d3aa3b65017 None admin] PATH_INFO: `/v3/projects/e178bea7e470433a8bb4389009f6ca45` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:15.927720 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f31c91-1906-4c45-8f7a-0d3aa3b65017 None admin] RBAC: Authorizing `identity:get_project(project_id=e178bea7e470433a8bb4389009f6ca45)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:15.929125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f31c91-1906-4c45-8f7a-0d3aa3b65017 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:15.930701 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 588/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:40:15 2022] GET /identity/v3/projects/e178bea7e470433a8bb4389009f6ca45 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:15.936944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfc0b7-9735-40c1-8ca1-b0a7aedd11b5 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:16.008613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:16.032607 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfc0b7-9735-40c1-8ca1-b0a7aedd11b5 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:16.065689 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd4484-1f50-442a-83b9-3f796feaa686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:16.066299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd4484-1f50-442a-83b9-3f796feaa686 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:16.066561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd4484-1f50-442a-83b9-3f796feaa686 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:16.066835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd4484-1f50-442a-83b9-3f796feaa686 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:16.067589 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd4484-1f50-442a-83b9-3f796feaa686 None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=e178bea7e470433a8bb4389009f6ca45, user.id=7f9da35db15a4b1398eb28c741c43764) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:40:16.067954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd4484-1f50-442a-83b9-3f796feaa686 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:16.069370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd4484-1f50-442a-83b9-3f796feaa686 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:16.074633 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 597/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:40:15 2022] GET /identity/v3/role_assignments?user.id=7f9da35db15a4b1398eb28c741c43764&scope.project.id=e178bea7e470433a8bb4389009f6ca45&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:17.935275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432429db-45fe-42a1-be23-35dcd197f59e None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:17.935815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432429db-45fe-42a1-be23-35dcd197f59e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:17.935815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432429db-45fe-42a1-be23-35dcd197f59e None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:17.937146 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 589/1186] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:17.942443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0e258-224f-4baf-85c6-56b78aefd7e9 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:17.942857 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0e258-224f-4baf-85c6-56b78aefd7e9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:17.942963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0e258-224f-4baf-85c6-56b78aefd7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:17.960684 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-efa0e258-224f-4baf-85c6-56b78aefd7e9 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:17.961847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-efa0e258-224f-4baf-85c6-56b78aefd7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:18.025481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 598/1187] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:40:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:40:18.289849 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432429db-45fe-42a1-be23-35dcd197f59e None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:18.324490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:18.389125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432429db-45fe-42a1-be23-35dcd197f59e None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:18.422506 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f48f9-3a4d-4de1-b33f-4af98c7911f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:18.423134 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f48f9-3a4d-4de1-b33f-4af98c7911f1 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:18.423420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f48f9-3a4d-4de1-b33f-4af98c7911f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:18.423691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f48f9-3a4d-4de1-b33f-4af98c7911f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:18.433719 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f48f9-3a4d-4de1-b33f-4af98c7911f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:40:18.438443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-435f48f9-3a4d-4de1-b33f-4af98c7911f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:40:18.438443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:40:18.438443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:40:18.438443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:40:18.438443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:40:18.438443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:40:18.438443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:40:18.438443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:40:18.438443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:40:18.438443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:40:18.438443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:40:18.438443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:40:18.438443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:40:18.438443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:40:18.438443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:40:18.438443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:40:18.439637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:40:18.439637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:40:18.439637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:40:18.439637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:40:18.439637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:40:18.439637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:40:18.439637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:40:18.439637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:40:18.439637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:40:18.439637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:40:18.439637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:40:18.439637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:40:18.439637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:40:18.439637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:40:18.439637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:40:18.439637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:40:18.440755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:40:18.440755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:40:18.440755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:40:18.440755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:40:18.440755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:40:18.440755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:40:18.440755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:40:18.440755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:40:18.440755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:40:18.440755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:40:18.440755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:40:18.440755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:40:18.440755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 590/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:40:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:40:18.445464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0e258-224f-4baf-85c6-56b78aefd7e9 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:18.509761 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:18.541392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0e258-224f-4baf-85c6-56b78aefd7e9 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:18.573849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650fe7d-1d57-4ac0-9d28-534b5b94a586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:18.574435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650fe7d-1d57-4ac0-9d28-534b5b94a586 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:18.574712 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650fe7d-1d57-4ac0-9d28-534b5b94a586 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:18.574982 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650fe7d-1d57-4ac0-9d28-534b5b94a586 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:18.575667 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650fe7d-1d57-4ac0-9d28-534b5b94a586 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:40:18.575969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650fe7d-1d57-4ac0-9d28-534b5b94a586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:18.577330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650fe7d-1d57-4ac0-9d28-534b5b94a586 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:18.585546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 599/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:40:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:18.591087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f48f9-3a4d-4de1-b33f-4af98c7911f1 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:18.681906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:18.688305 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f48f9-3a4d-4de1-b33f-4af98c7911f1 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:18.721753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf9822-d372-4fd7-9d64-362689145aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:18.722359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf9822-d372-4fd7-9d64-362689145aa8 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:18.722636 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf9822-d372-4fd7-9d64-362689145aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:18.722906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf9822-d372-4fd7-9d64-362689145aa8 None admin] PATH_INFO: `/v3/users/7f9da35db15a4b1398eb28c741c43764` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:18.724067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf9822-d372-4fd7-9d64-362689145aa8 None admin] RBAC: Authorizing `identity:get_user(user_id=7f9da35db15a4b1398eb28c741c43764)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:18.725760 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf9822-d372-4fd7-9d64-362689145aa8 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:18.727321 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 591/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:40:18 2022] GET /identity/v3/users/7f9da35db15a4b1398eb28c741c43764 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:18.732877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650fe7d-1d57-4ac0-9d28-534b5b94a586 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:18.829324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650fe7d-1d57-4ac0-9d28-534b5b94a586 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:18.856087 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:18.861558 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d934814-0e4b-4040-b991-15174a30f1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:18.862167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d934814-0e4b-4040-b991-15174a30f1a4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:18.862445 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d934814-0e4b-4040-b991-15174a30f1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:18.862715 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d934814-0e4b-4040-b991-15174a30f1a4 None admin] PATH_INFO: `/v3/projects/e178bea7e470433a8bb4389009f6ca45` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:18.863832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d934814-0e4b-4040-b991-15174a30f1a4 None admin] RBAC: Authorizing `identity:get_project(project_id=e178bea7e470433a8bb4389009f6ca45)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:18.868833 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d934814-0e4b-4040-b991-15174a30f1a4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:18.868833 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 600/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:40:18 2022] GET /identity/v3/projects/e178bea7e470433a8bb4389009f6ca45 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:18.873346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf9822-d372-4fd7-9d64-362689145aa8 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:18.982763 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf9822-d372-4fd7-9d64-362689145aa8 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:19.014921 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dcb3dd-a576-4285-a0bb-2f9aaf70f856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:19.015583 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcb3dd-a576-4285-a0bb-2f9aaf70f856 None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:19.015859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcb3dd-a576-4285-a0bb-2f9aaf70f856 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:19.016129 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcb3dd-a576-4285-a0bb-2f9aaf70f856 None admin] PATH_INFO: `/v3/projects/e178bea7e470433a8bb4389009f6ca45/users/7f9da35db15a4b1398eb28c741c43764/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:19.017978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dcb3dd-a576-4285-a0bb-2f9aaf70f856 None admin] RBAC: Authorizing `identity:create_grant(project_id=e178bea7e470433a8bb4389009f6ca45, user_id=7f9da35db15a4b1398eb28c741c43764, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:19.019418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dcb3dd-a576-4285-a0bb-2f9aaf70f856 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:19.102470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 592/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:40:18 2022] PUT /identity/v3/projects/e178bea7e470433a8bb4389009f6ca45/users/7f9da35db15a4b1398eb28c741c43764/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:40:20.980068 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c46bde-20bc-4ee6-8dbf-c036dea3fd61 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:20.980613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c46bde-20bc-4ee6-8dbf-c036dea3fd61 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:20.980613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c46bde-20bc-4ee6-8dbf-c036dea3fd61 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:20.981958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 601/1193] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:20.987488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489ff88-d338-4f01-8bf2-516f0095bf99 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:20.987926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489ff88-d338-4f01-8bf2-516f0095bf99 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:20.988032 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489ff88-d338-4f01-8bf2-516f0095bf99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:21.006139 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-8489ff88-d338-4f01-8bf2-516f0095bf99 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:21.007330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8489ff88-d338-4f01-8bf2-516f0095bf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:21.023938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:21.072104 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 593/1194] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:40:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:40:21.337829 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c46bde-20bc-4ee6-8dbf-c036dea3fd61 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:21.434791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c46bde-20bc-4ee6-8dbf-c036dea3fd61 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:21.466992 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935bbee-ab6f-424c-90b5-2f0f58538078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:21.467630 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935bbee-ab6f-424c-90b5-2f0f58538078 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:21.467907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935bbee-ab6f-424c-90b5-2f0f58538078 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:21.468180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935bbee-ab6f-424c-90b5-2f0f58538078 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:21.477874 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935bbee-ab6f-424c-90b5-2f0f58538078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:40:21.482591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-5935bbee-ab6f-424c-90b5-2f0f58538078 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:40:21.482591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:40:21.482591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:40:21.482591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:40:21.482591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:40:21.482591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:40:21.482591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:40:21.482591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:40:21.482591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:40:21.482591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:40:21.482591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:40:21.482591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:40:21.482591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:40:21.482591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:40:21.482591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:40:21.482591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:40:21.483714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:40:21.483714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:40:21.483714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:40:21.483714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:40:21.483714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:40:21.483714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:40:21.483714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:40:21.483714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:40:21.483714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:40:21.483714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:40:21.483714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:40:21.483714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:40:21.483714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:40:21.483714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:40:21.483714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:40:21.483714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:40:21.484543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:40:21.484543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:40:21.484543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:40:21.484543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:40:21.484543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:40:21.484543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:40:21.484543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:40:21.484543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:40:21.484543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:40:21.484543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:40:21.484543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:40:21.484543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:40:21.484543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 602/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:40:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:40:21.489932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489ff88-d338-4f01-8bf2-516f0095bf99 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:21.585614 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489ff88-d338-4f01-8bf2-516f0095bf99 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:21.618950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df8aac-9983-4810-86a5-7c12f33f9298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:21.619560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df8aac-9983-4810-86a5-7c12f33f9298 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:21.619837 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df8aac-9983-4810-86a5-7c12f33f9298 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:21.620107 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df8aac-9983-4810-86a5-7c12f33f9298 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:21.620763 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df8aac-9983-4810-86a5-7c12f33f9298 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:40:21.621064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df8aac-9983-4810-86a5-7c12f33f9298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:21.622417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df8aac-9983-4810-86a5-7c12f33f9298 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:21.630184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 594/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:40:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:21.635868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935bbee-ab6f-424c-90b5-2f0f58538078 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:21.692921 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:21.732576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935bbee-ab6f-424c-90b5-2f0f58538078 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:21.764927 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49499f-e510-4ccd-b98a-8a87fd011a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:21.765528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49499f-e510-4ccd-b98a-8a87fd011a18 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:21.765804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49499f-e510-4ccd-b98a-8a87fd011a18 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:21.766071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49499f-e510-4ccd-b98a-8a87fd011a18 None admin] PATH_INFO: `/v3/users/7f9da35db15a4b1398eb28c741c43764` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:21.767236 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d49499f-e510-4ccd-b98a-8a87fd011a18 None admin] RBAC: Authorizing `identity:get_user(user_id=7f9da35db15a4b1398eb28c741c43764)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:21.768660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d49499f-e510-4ccd-b98a-8a87fd011a18 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:21.770195 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 603/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Aug 30 14:40:21 2022] GET /identity/v3/users/7f9da35db15a4b1398eb28c741c43764 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:21.776176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df8aac-9983-4810-86a5-7c12f33f9298 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:21.792240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:21.873692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df8aac-9983-4810-86a5-7c12f33f9298 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:21.906552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575accc-80fe-435c-a8e8-ff8e5e5eaf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:21.907163 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575accc-80fe-435c-a8e8-ff8e5e5eaf6e None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:21.907441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575accc-80fe-435c-a8e8-ff8e5e5eaf6e None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:21.907709 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575accc-80fe-435c-a8e8-ff8e5e5eaf6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:21.973278 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575accc-80fe-435c-a8e8-ff8e5e5eaf6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:21.975288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575accc-80fe-435c-a8e8-ff8e5e5eaf6e None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:21.988730 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:22.069766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 595/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:40:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:40:22.075778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49499f-e510-4ccd-b98a-8a87fd011a18 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:22.171477 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49499f-e510-4ccd-b98a-8a87fd011a18 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:22.195535 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:22.203676 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f417db-1652-4d49-bc43-560660dc8c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:22.204279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f417db-1652-4d49-bc43-560660dc8c2f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:22.208599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f417db-1652-4d49-bc43-560660dc8c2f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:22.208599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f417db-1652-4d49-bc43-560660dc8c2f None admin] PATH_INFO: `/v3/projects/e178bea7e470433a8bb4389009f6ca45` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:22.208599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f417db-1652-4d49-bc43-560660dc8c2f None admin] RBAC: Authorizing `identity:get_project(project_id=e178bea7e470433a8bb4389009f6ca45)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:22.208599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f417db-1652-4d49-bc43-560660dc8c2f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:22.209406 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 604/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Aug 30 14:40:22 2022] GET /identity/v3/projects/e178bea7e470433a8bb4389009f6ca45 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:22.215384 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575accc-80fe-435c-a8e8-ff8e5e5eaf6e None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:22.312252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575accc-80fe-435c-a8e8-ff8e5e5eaf6e None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:22.325861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:22.346859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767b639-8cc8-4380-9a46-7eaba00471a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:22.347479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767b639-8cc8-4380-9a46-7eaba00471a4 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:22.347763 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767b639-8cc8-4380-9a46-7eaba00471a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:22.348036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767b639-8cc8-4380-9a46-7eaba00471a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:22.349278 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767b639-8cc8-4380-9a46-7eaba00471a4 None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=e178bea7e470433a8bb4389009f6ca45, user.id=7f9da35db15a4b1398eb28c741c43764) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:40:22.349585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767b639-8cc8-4380-9a46-7eaba00471a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:22.351029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767b639-8cc8-4380-9a46-7eaba00471a4 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:22.356659 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 596/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:40:22 2022] GET /identity/v3/role_assignments?user.id=7f9da35db15a4b1398eb28c741c43764&scope.project.id=e178bea7e470433a8bb4389009f6ca45&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:32.228037 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:32.359458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=11 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:35.203714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d637d42-65dc-4d89-889a-7bad81a08529 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:35.204432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d637d42-65dc-4d89-889a-7bad81a08529 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:35.204432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d637d42-65dc-4d89-889a-7bad81a08529 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:35.206203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 605/1201] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:41:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:41:35.211322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd287c67-5c5f-4677-9c78-44f486249e08 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:35.211762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd287c67-5c5f-4677-9c78-44f486249e08 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:35.211861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd287c67-5c5f-4677-9c78-44f486249e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:35.229897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-fd287c67-5c5f-4677-9c78-44f486249e08 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:41:35.231086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-fd287c67-5c5f-4677-9c78-44f486249e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:41:35.297532 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 597/1202] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:41:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:41:35.568781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d637d42-65dc-4d89-889a-7bad81a08529 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:35.666893 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d637d42-65dc-4d89-889a-7bad81a08529 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:35.700401 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63db2d6-4fb9-4a99-a003-33081768519d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:35.701084 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63db2d6-4fb9-4a99-a003-33081768519d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:35.701430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63db2d6-4fb9-4a99-a003-33081768519d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:35.701708 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63db2d6-4fb9-4a99-a003-33081768519d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:35.714644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63db2d6-4fb9-4a99-a003-33081768519d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:35.716141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63db2d6-4fb9-4a99-a003-33081768519d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:35.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-d63db2d6-4fb9-4a99-a003-33081768519d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:35.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:35.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:35.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:35.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:35.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:35.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:35.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:35.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:35.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:35.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:35.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:35.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:41:35.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:41:35.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:41:35.731250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:35.735652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:41:35.737486 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:41:35.737486 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:35.737486 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:41:35.737486 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 606/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:35.739286 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd287c67-5c5f-4677-9c78-44f486249e08 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:35.835632 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd287c67-5c5f-4677-9c78-44f486249e08 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:35.863885 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:35.869467 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f2a76-d265-4afd-afb5-b02371fefa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:35.870080 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f2a76-d265-4afd-afb5-b02371fefa26 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:35.870364 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f2a76-d265-4afd-afb5-b02371fefa26 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:35.870635 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f2a76-d265-4afd-afb5-b02371fefa26 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:35.871307 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f2a76-d265-4afd-afb5-b02371fefa26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:35.871647 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f2a76-d265-4afd-afb5-b02371fefa26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:35.873003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f2a76-d265-4afd-afb5-b02371fefa26 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:35.883929 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 598/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:35.892730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63db2d6-4fb9-4a99-a003-33081768519d None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:35.946493 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:35.992443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63db2d6-4fb9-4a99-a003-33081768519d None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:36.025011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe681f87-6f16-4db1-8287-3519c1d9d12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:36.026278 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe681f87-6f16-4db1-8287-3519c1d9d12d None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:36.026278 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe681f87-6f16-4db1-8287-3519c1d9d12d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:36.026278 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe681f87-6f16-4db1-8287-3519c1d9d12d None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:36.026796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe681f87-6f16-4db1-8287-3519c1d9d12d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:36.028259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe681f87-6f16-4db1-8287-3519c1d9d12d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:36.130784 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 607/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Aug 30 14:41:35 2022] POST /identity/v3/users => generated 251 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:41:38.063065 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9bf4e9-b444-47d7-91ed-a8ef4f00ca5b None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:38.063637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9bf4e9-b444-47d7-91ed-a8ef4f00ca5b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:38.063637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9bf4e9-b444-47d7-91ed-a8ef4f00ca5b None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:38.065558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 599/1206] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:41:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:41:38.070544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa66fc-0b58-4940-b82f-ace6815f1f05 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:38.070957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa66fc-0b58-4940-b82f-ace6815f1f05 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:38.071091 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa66fc-0b58-4940-b82f-ace6815f1f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:38.089798 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1caa66fc-0b58-4940-b82f-ace6815f1f05 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:41:38.090995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1caa66fc-0b58-4940-b82f-ace6815f1f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:41:38.157238 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 608/1207] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:41:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:41:38.427449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9bf4e9-b444-47d7-91ed-a8ef4f00ca5b None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:38.526391 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9bf4e9-b444-47d7-91ed-a8ef4f00ca5b None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:38.559654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa81c81-e430-4af4-aaec-eacc85c85f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:38.561543 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81c81-e430-4af4-aaec-eacc85c85f81 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:38.561543 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81c81-e430-4af4-aaec-eacc85c85f81 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:38.561543 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81c81-e430-4af4-aaec-eacc85c85f81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:38.570570 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa81c81-e430-4af4-aaec-eacc85c85f81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:41:38.575336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-6aa81c81-e430-4af4-aaec-eacc85c85f81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:41:38.575336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:38.575336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:38.575336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:38.575336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:38.575336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:38.575336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:38.575336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:38.575336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:38.575336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:38.575336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:38.575336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:38.575336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:41:38.575336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:41:38.575336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:41:38.575336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:41:38.576360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:41:38.576360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:41:38.576360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:41:38.576360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:41:38.576360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:38.576360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:38.576360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:38.576360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:38.576360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:38.576360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:38.576360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:38.576360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:38.576360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:38.576360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:38.576360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:38.576360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:38.577299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:38.577299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:38.577299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:38.577299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:38.577299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:41:38.577299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:41:38.577299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:41:38.577299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:41:38.577299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:41:38.577299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:41:38.577299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:41:38.577299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:41:38.577299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 600/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:41:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:38.583309 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa66fc-0b58-4940-b82f-ace6815f1f05 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:38.621231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:38.678365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa66fc-0b58-4940-b82f-ace6815f1f05 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:38.711392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2580a28-9d4d-47ce-b649-8480ae276d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:38.711986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2580a28-9d4d-47ce-b649-8480ae276d66 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:38.712260 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2580a28-9d4d-47ce-b649-8480ae276d66 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:38.712525 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2580a28-9d4d-47ce-b649-8480ae276d66 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:38.713180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2580a28-9d4d-47ce-b649-8480ae276d66 None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:38.713475 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2580a28-9d4d-47ce-b649-8480ae276d66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:38.714833 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2580a28-9d4d-47ce-b649-8480ae276d66 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:38.722966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 609/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:41:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:38.728778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa81c81-e430-4af4-aaec-eacc85c85f81 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:38.780911 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:38.829014 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa81c81-e430-4af4-aaec-eacc85c85f81 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:38.861787 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dd452-1463-4e16-93b4-d10b3f201997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:38.862390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dd452-1463-4e16-93b4-d10b3f201997 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:38.862663 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dd452-1463-4e16-93b4-d10b3f201997 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:38.862927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dd452-1463-4e16-93b4-d10b3f201997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:38.872953 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2dd452-1463-4e16-93b4-d10b3f201997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:38.874378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2dd452-1463-4e16-93b4-d10b3f201997 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:38.888681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-1e2dd452-1463-4e16-93b4-d10b3f201997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:38.888681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:38.888681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:38.888681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:38.888681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:38.888681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:38.888681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:38.888681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:38.888681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:38.888681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:38.888681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:38.888681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:38.888681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:41:38.888681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:41:38.888681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:41:38.888681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:38.893306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:41:38.894200 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:41:38.894200 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:38.894200 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:41:38.894200 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 601/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:38.896821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2580a28-9d4d-47ce-b649-8480ae276d66 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:38.992148 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:38.994927 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2580a28-9d4d-47ce-b649-8480ae276d66 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:39.028162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e7fce-ad30-4f71-ac2a-b62ba93c5500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:39.028751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e7fce-ad30-4f71-ac2a-b62ba93c5500 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:39.029025 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e7fce-ad30-4f71-ac2a-b62ba93c5500 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:39.029293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e7fce-ad30-4f71-ac2a-b62ba93c5500 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:39.029922 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e7fce-ad30-4f71-ac2a-b62ba93c5500 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:39.030225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e7fce-ad30-4f71-ac2a-b62ba93c5500 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:39.031607 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e7fce-ad30-4f71-ac2a-b62ba93c5500 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:39.042198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 610/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:39.048138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dd452-1463-4e16-93b4-d10b3f201997 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:39.151821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dd452-1463-4e16-93b4-d10b3f201997 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:39.153287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:39.185297 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ebe82-dd20-4837-8e8f-38bb751a4823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:39.185955 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ebe82-dd20-4837-8e8f-38bb751a4823 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:39.186243 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ebe82-dd20-4837-8e8f-38bb751a4823 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:39.186515 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ebe82-dd20-4837-8e8f-38bb751a4823 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:39.193856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ebe82-dd20-4837-8e8f-38bb751a4823 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:39.196169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ebe82-dd20-4837-8e8f-38bb751a4823 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:39.206833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-215ebe82-dd20-4837-8e8f-38bb751a4823 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:41:39.206833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:39.206833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:39.206833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:39.206833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:39.206833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:39.206833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:39.206833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:39.206833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:39.206833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:39.206833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:39.206833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:39.206833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:41:39.206833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:41:39.206833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:41:39.206833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:39.208006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:39.209131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:39.209131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:39.209131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:41:39.209131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:41:39.209131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:41:39.209131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:41:39.209131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:41:39.209131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:41:39.209131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:41:39.209131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:41:39.209131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 602/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:39.213809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e7fce-ad30-4f71-ac2a-b62ba93c5500 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:39.311374 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e7fce-ad30-4f71-ac2a-b62ba93c5500 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:39.344088 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3f05c-de0b-44b1-b5bf-bfc8566ec477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:39.344687 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3f05c-de0b-44b1-b5bf-bfc8566ec477 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:39.344968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3f05c-de0b-44b1-b5bf-bfc8566ec477 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:39.345241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3f05c-de0b-44b1-b5bf-bfc8566ec477 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:39.346004 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d3f05c-de0b-44b1-b5bf-bfc8566ec477 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:39.346304 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d3f05c-de0b-44b1-b5bf-bfc8566ec477 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:39.347768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d3f05c-de0b-44b1-b5bf-bfc8566ec477 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:39.365159 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 611/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Aug 30 14:41:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:39.371068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ebe82-dd20-4837-8e8f-38bb751a4823 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:39.471058 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ebe82-dd20-4837-8e8f-38bb751a4823 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:39.504160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2314fd-9580-42bc-9da2-f5d7e6b89eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:39.504754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2314fd-9580-42bc-9da2-f5d7e6b89eac None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:39.505030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2314fd-9580-42bc-9da2-f5d7e6b89eac None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:39.505299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2314fd-9580-42bc-9da2-f5d7e6b89eac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:39.534550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:39.571597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2314fd-9580-42bc-9da2-f5d7e6b89eac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:39.573203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2314fd-9580-42bc-9da2-f5d7e6b89eac None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:39.670354 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 603/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:41:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 14:41:39.676795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3f05c-de0b-44b1-b5bf-bfc8566ec477 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:39.695161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:39.773949 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3f05c-de0b-44b1-b5bf-bfc8566ec477 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:39.807776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9d483-0fad-4b8d-803d-608d07029915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:39.808339 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9d483-0fad-4b8d-803d-608d07029915 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:39.808618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9d483-0fad-4b8d-803d-608d07029915 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:39.808894 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9d483-0fad-4b8d-803d-608d07029915 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:39.818760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9d483-0fad-4b8d-803d-608d07029915 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:39.820220 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9d483-0fad-4b8d-803d-608d07029915 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:39.834154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-abc9d483-0fad-4b8d-803d-608d07029915 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:39.834154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:39.834154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:39.834154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:39.834154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:39.834154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:39.834154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:39.834154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:39.834154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:39.834154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:39.834154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:39.834154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:39.834154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:41:39.834154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:41:39.834154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:41:39.834154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:39.835282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:41:39.836562 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:41:39.836562 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:39.836562 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:41:39.836562 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 612/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:39.841142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2314fd-9580-42bc-9da2-f5d7e6b89eac None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:39.901749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:39.938358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2314fd-9580-42bc-9da2-f5d7e6b89eac None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:39.971594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2daf2-448f-4876-a015-70b7b9892c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:39.972182 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2daf2-448f-4876-a015-70b7b9892c04 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:39.972455 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2daf2-448f-4876-a015-70b7b9892c04 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:39.972720 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2daf2-448f-4876-a015-70b7b9892c04 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:39.973349 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2daf2-448f-4876-a015-70b7b9892c04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:39.973649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2daf2-448f-4876-a015-70b7b9892c04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:39.975165 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2daf2-448f-4876-a015-70b7b9892c04 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:39.987396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 604/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:39.993073 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9d483-0fad-4b8d-803d-608d07029915 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:40.060903 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:40.091222 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9d483-0fad-4b8d-803d-608d07029915 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:40.125932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b000338-85ce-4ebd-9448-e25ce6be86d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:40.126557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b000338-85ce-4ebd-9448-e25ce6be86d8 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:40.126836 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b000338-85ce-4ebd-9448-e25ce6be86d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:40.127123 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b000338-85ce-4ebd-9448-e25ce6be86d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:40.133571 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b000338-85ce-4ebd-9448-e25ce6be86d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:40.134997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b000338-85ce-4ebd-9448-e25ce6be86d8 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:40.146690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-2b000338-85ce-4ebd-9448-e25ce6be86d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:41:40.146690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:40.146690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:40.146690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:40.146690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:40.146690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:40.146690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:40.146690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:40.146690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:40.146690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:40.146690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:40.146690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:40.146690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:41:40.146690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:41:40.146690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:40.150180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:40.151307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:41:40.151307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:41:40.151307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:41:40.151307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:41:40.151307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:41:40.151307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:41:40.151307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:41:40.151307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:41:40.151307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 613/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:41:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:40.153658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2daf2-448f-4876-a015-70b7b9892c04 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:40.252180 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2daf2-448f-4876-a015-70b7b9892c04 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:40.275843 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:40.287352 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb575a28-7b14-4c7b-882b-38c71b78b91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:40.287952 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb575a28-7b14-4c7b-882b-38c71b78b91c None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:40.288229 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb575a28-7b14-4c7b-882b-38c71b78b91c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:40.288535 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb575a28-7b14-4c7b-882b-38c71b78b91c None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:40.289223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb575a28-7b14-4c7b-882b-38c71b78b91c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:40.289524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb575a28-7b14-4c7b-882b-38c71b78b91c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:40.290939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb575a28-7b14-4c7b-882b-38c71b78b91c None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:40.302087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 605/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:41:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:40.308170 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b000338-85ce-4ebd-9448-e25ce6be86d8 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:40.406759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b000338-85ce-4ebd-9448-e25ce6be86d8 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:40.433414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:40.439622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be75e0-7348-418f-a0a1-3cfabf6c33df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:40.440231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be75e0-7348-418f-a0a1-3cfabf6c33df None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:40.440510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be75e0-7348-418f-a0a1-3cfabf6c33df None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:40.440784 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be75e0-7348-418f-a0a1-3cfabf6c33df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:40.441517 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be75e0-7348-418f-a0a1-3cfabf6c33df None admin] RBAC: Adding query filter params (role.id=244ad93ecd1949c9a6e8f74b135b43b4, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=f4f11655cd51438e9652869f74ffa975) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:40.441826 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be75e0-7348-418f-a0a1-3cfabf6c33df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:40.443299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be75e0-7348-418f-a0a1-3cfabf6c33df None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:40.448678 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 614/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:41:40 2022] GET /identity/v3/role_assignments?user.id=f4f11655cd51438e9652869f74ffa975&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=244ad93ecd1949c9a6e8f74b135b43b4 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:41:42.293738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d847b-f88f-4cb0-a11f-e03bfd4c4490 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:42.294220 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d847b-f88f-4cb0-a11f-e03bfd4c4490 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:42.294354 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d847b-f88f-4cb0-a11f-e03bfd4c4490 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:42.295698 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 606/1220] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:41:42.300973 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34cc58-dac6-4833-a616-438a1001d23b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:42.301368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34cc58-dac6-4833-a616-438a1001d23b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:42.301549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34cc58-dac6-4833-a616-438a1001d23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:42.319497 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2b34cc58-dac6-4833-a616-438a1001d23b None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:41:42.320758 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2b34cc58-dac6-4833-a616-438a1001d23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:41:42.385876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 615/1221] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:41:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:41:42.657052 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d847b-f88f-4cb0-a11f-e03bfd4c4490 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:42.756798 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d847b-f88f-4cb0-a11f-e03bfd4c4490 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:42.790714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4c4c8-1b6b-4268-96d9-a2e10aafcb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:42.791356 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4c4c8-1b6b-4268-96d9-a2e10aafcb00 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:42.791638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4c4c8-1b6b-4268-96d9-a2e10aafcb00 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:42.791908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4c4c8-1b6b-4268-96d9-a2e10aafcb00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:42.801808 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4c4c8-1b6b-4268-96d9-a2e10aafcb00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:41:42.806504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-ccc4c4c8-1b6b-4268-96d9-a2e10aafcb00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:41:42.806504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:42.806504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:42.806504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:42.806504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:42.806504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:42.806504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:42.806504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:42.806504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:42.806504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:42.806504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:42.806504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:42.806504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:41:42.806504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:41:42.806504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:41:42.806504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:41:42.807696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:41:42.807696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:41:42.807696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:41:42.807696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:41:42.807696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:42.807696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:42.807696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:42.807696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:42.807696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:42.807696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:42.807696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:42.807696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:42.807696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:42.807696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:42.807696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:42.807696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:42.808597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:42.808597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:42.808597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:42.808597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:42.808597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:41:42.808597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:41:42.808597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:41:42.808597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:41:42.808597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:41:42.808597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:41:42.808597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:41:42.808597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:41:42.808597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 607/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:41:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:42.813616 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34cc58-dac6-4833-a616-438a1001d23b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:42.910641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34cc58-dac6-4833-a616-438a1001d23b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:42.933275 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:42.944053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a072f76-9d2b-4018-b480-42a9aefe62c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:42.944650 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a072f76-9d2b-4018-b480-42a9aefe62c0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:42.944926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a072f76-9d2b-4018-b480-42a9aefe62c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:42.945197 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a072f76-9d2b-4018-b480-42a9aefe62c0 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:42.945856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a072f76-9d2b-4018-b480-42a9aefe62c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:42.946148 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a072f76-9d2b-4018-b480-42a9aefe62c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:42.947552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a072f76-9d2b-4018-b480-42a9aefe62c0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:42.955051 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 616/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:41:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:42.960847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4c4c8-1b6b-4268-96d9-a2e10aafcb00 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:43.005379 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:43.061624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4c4c8-1b6b-4268-96d9-a2e10aafcb00 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:43.094665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b04f5-79cf-4334-a291-5e8fcf242738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:43.095378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b04f5-79cf-4334-a291-5e8fcf242738 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:43.095655 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b04f5-79cf-4334-a291-5e8fcf242738 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:43.095920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b04f5-79cf-4334-a291-5e8fcf242738 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:43.105496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b04f5-79cf-4334-a291-5e8fcf242738 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:43.106941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b04f5-79cf-4334-a291-5e8fcf242738 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:43.122378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-ea1b04f5-79cf-4334-a291-5e8fcf242738 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:43.122378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:43.122378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:43.122378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:43.122378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:43.122378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:43.122378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:43.122378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:43.122378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:43.122378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:43.122378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:43.122378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:43.122378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:41:43.122378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:41:43.122378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:41:43.122378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:43.123382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:41:43.124620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:41:43.124620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:43.124620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:41:43.124620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 608/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:43.129850 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a072f76-9d2b-4018-b480-42a9aefe62c0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:43.229380 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a072f76-9d2b-4018-b480-42a9aefe62c0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:43.262414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50c512-9af8-4947-accf-fe9ceb959261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:43.263015 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50c512-9af8-4947-accf-fe9ceb959261 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:43.263320 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50c512-9af8-4947-accf-fe9ceb959261 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:43.263593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50c512-9af8-4947-accf-fe9ceb959261 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:43.264238 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f50c512-9af8-4947-accf-fe9ceb959261 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:43.264543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f50c512-9af8-4947-accf-fe9ceb959261 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:43.265942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f50c512-9af8-4947-accf-fe9ceb959261 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:43.277762 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 617/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:43.283875 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b04f5-79cf-4334-a291-5e8fcf242738 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:43.376354 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:43.383022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b04f5-79cf-4334-a291-5e8fcf242738 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:43.416912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70f18f-3531-4838-a984-b2429dd3bcf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:43.417553 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70f18f-3531-4838-a984-b2429dd3bcf7 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:43.417883 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70f18f-3531-4838-a984-b2429dd3bcf7 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:43.418158 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70f18f-3531-4838-a984-b2429dd3bcf7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:43.425541 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70f18f-3531-4838-a984-b2429dd3bcf7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:43.426985 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70f18f-3531-4838-a984-b2429dd3bcf7 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:43.438302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-1d70f18f-3531-4838-a984-b2429dd3bcf7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:41:43.438302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:43.438302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:43.438302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:43.438302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:43.438302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:43.438302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:43.438302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:43.438302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:43.438302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:43.438302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:43.438302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:43.438302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:41:43.438302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:41:43.438302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:41:43.438302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:43.439521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:43.440714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:43.440714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:43.440714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:41:43.440714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:41:43.440714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:41:43.440714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:41:43.440714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:41:43.440714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:41:43.440714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:41:43.440714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:41:43.440714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 609/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:43.445397 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50c512-9af8-4947-accf-fe9ceb959261 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:43.491831 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:43.542960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50c512-9af8-4947-accf-fe9ceb959261 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:43.575602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0ebea-14d5-401f-a3de-b82204546911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:43.576198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0ebea-14d5-401f-a3de-b82204546911 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:43.576473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0ebea-14d5-401f-a3de-b82204546911 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:43.576741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0ebea-14d5-401f-a3de-b82204546911 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:43.577462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c0ebea-14d5-401f-a3de-b82204546911 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:43.577763 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c0ebea-14d5-401f-a3de-b82204546911 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:43.579202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c0ebea-14d5-401f-a3de-b82204546911 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:43.596276 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 618/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Aug 30 14:41:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:43.601947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70f18f-3531-4838-a984-b2429dd3bcf7 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:43.700903 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70f18f-3531-4838-a984-b2429dd3bcf7 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:43.734846 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20da9f-20a1-421a-949f-8d0b2acf07fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:43.735499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20da9f-20a1-421a-949f-8d0b2acf07fe None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:43.735778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20da9f-20a1-421a-949f-8d0b2acf07fe None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:43.736050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20da9f-20a1-421a-949f-8d0b2acf07fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:43.746490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20da9f-20a1-421a-949f-8d0b2acf07fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:43.747888 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20da9f-20a1-421a-949f-8d0b2acf07fe None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:43.761824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-fb20da9f-20a1-421a-949f-8d0b2acf07fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:43.761824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:43.761824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:43.761824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:43.761824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:43.761824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:43.761824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:43.761824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:43.761824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:43.761824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:43.761824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:43.761824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:43.761824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:41:43.761824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:41:43.761824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:41:43.761824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:43.763036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:41:43.764204 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:41:43.764204 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:43.764204 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:41:43.764204 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 610/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:43.769632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0ebea-14d5-401f-a3de-b82204546911 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:43.866943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0ebea-14d5-401f-a3de-b82204546911 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:43.869214 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:43.900307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e39ac-e241-42fe-974d-d03df9a27256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:43.900907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e39ac-e241-42fe-974d-d03df9a27256 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:43.901186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e39ac-e241-42fe-974d-d03df9a27256 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:43.901451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e39ac-e241-42fe-974d-d03df9a27256 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:43.902089 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e39ac-e241-42fe-974d-d03df9a27256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:43.902387 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e39ac-e241-42fe-974d-d03df9a27256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:43.903790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e39ac-e241-42fe-974d-d03df9a27256 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:43.915474 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 619/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:43.921278 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20da9f-20a1-421a-949f-8d0b2acf07fe None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:43.930625 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:44.020325 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20da9f-20a1-421a-949f-8d0b2acf07fe None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:44.052510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf920fe-2237-47ee-9e4b-c7d6ee06f4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:44.053128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf920fe-2237-47ee-9e4b-c7d6ee06f4ee None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:44.053406 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf920fe-2237-47ee-9e4b-c7d6ee06f4ee None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:44.053677 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf920fe-2237-47ee-9e4b-c7d6ee06f4ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:44.060159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf920fe-2237-47ee-9e4b-c7d6ee06f4ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:44.061589 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf920fe-2237-47ee-9e4b-c7d6ee06f4ee None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:44.073375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-0cf920fe-2237-47ee-9e4b-c7d6ee06f4ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:41:44.073375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:44.073375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:44.073375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:44.073375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:44.073375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:44.073375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:44.073375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:44.073375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:44.073375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:44.073375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:44.073375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:44.073375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:41:44.073375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:41:44.073375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:44.074417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:44.075564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:41:44.075564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:41:44.075564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:41:44.075564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:41:44.075564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:41:44.075564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:41:44.075564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:41:44.075564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:41:44.075564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 611/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:41:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:44.080887 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e39ac-e241-42fe-974d-d03df9a27256 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:44.178823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e39ac-e241-42fe-974d-d03df9a27256 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:44.216706 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428effa-78b6-449f-a5e3-39289dce1e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:44.217310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428effa-78b6-449f-a5e3-39289dce1e36 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:44.217584 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428effa-78b6-449f-a5e3-39289dce1e36 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:44.217854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428effa-78b6-449f-a5e3-39289dce1e36 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:44.218543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428effa-78b6-449f-a5e3-39289dce1e36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:44.218886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428effa-78b6-449f-a5e3-39289dce1e36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:44.220307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428effa-78b6-449f-a5e3-39289dce1e36 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:44.231852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 620/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:41:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:44.237970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf920fe-2237-47ee-9e4b-c7d6ee06f4ee None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:44.303834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:44.338825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf920fe-2237-47ee-9e4b-c7d6ee06f4ee None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:44.370673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58578c0-24f4-4ffa-a76d-f2e24760e499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:44.371337 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58578c0-24f4-4ffa-a76d-f2e24760e499 None admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:44.371610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58578c0-24f4-4ffa-a76d-f2e24760e499 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:44.371876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58578c0-24f4-4ffa-a76d-f2e24760e499 None admin] PATH_INFO: `/v3/projects/1147574d59c7404baf76a24149684f6e/users/f4f11655cd51438e9652869f74ffa975/roles/244ad93ecd1949c9a6e8f74b135b43b4` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:44.385963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58578c0-24f4-4ffa-a76d-f2e24760e499 None admin] RBAC: Authorizing `identity:create_grant(project_id=1147574d59c7404baf76a24149684f6e, user_id=f4f11655cd51438e9652869f74ffa975, role_id=244ad93ecd1949c9a6e8f74b135b43b4)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:44.387425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58578c0-24f4-4ffa-a76d-f2e24760e499 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:44.467213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 612/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:41:44 2022] PUT /identity/v3/projects/1147574d59c7404baf76a24149684f6e/users/f4f11655cd51438e9652869f74ffa975/roles/244ad93ecd1949c9a6e8f74b135b43b4 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:41:46.331462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5233f-645b-4c8f-afa7-647d1f2c76f1 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:46.331978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5233f-645b-4c8f-afa7-647d1f2c76f1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:46.332095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5233f-645b-4c8f-afa7-647d1f2c76f1 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:46.333351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 621/1233] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:41:46.339031 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174b35f-e58e-4ea7-8637-32ed953a8613 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:46.339680 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174b35f-e58e-4ea7-8637-32ed953a8613 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:46.339680 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174b35f-e58e-4ea7-8637-32ed953a8613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:46.361408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-3174b35f-e58e-4ea7-8637-32ed953a8613 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:41:46.362598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-3174b35f-e58e-4ea7-8637-32ed953a8613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:41:46.429784 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 613/1234] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:41:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:41:46.699905 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5233f-645b-4c8f-afa7-647d1f2c76f1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:46.725734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:46.803611 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5233f-645b-4c8f-afa7-647d1f2c76f1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:46.839081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cad35-dd24-4f22-8c8f-02d045119562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:46.839730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cad35-dd24-4f22-8c8f-02d045119562 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:46.840009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cad35-dd24-4f22-8c8f-02d045119562 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:46.840278 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cad35-dd24-4f22-8c8f-02d045119562 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:46.851039 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cad35-dd24-4f22-8c8f-02d045119562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:41:46.855737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-dd0cad35-dd24-4f22-8c8f-02d045119562 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:41:46.855737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:46.855737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:46.855737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:46.855737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:46.855737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:46.855737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:46.855737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:46.855737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:46.855737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:46.855737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:46.855737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:46.855737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:41:46.855737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:41:46.855737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:41:46.855737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:41:46.856818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:41:46.856818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:41:46.856818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:41:46.856818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:41:46.856818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:46.856818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:46.856818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:46.856818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:46.856818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:46.856818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:46.856818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:46.856818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:46.856818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:46.856818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:46.856818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:46.856818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:46.858052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:46.858052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:46.858052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:46.858052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:46.858052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:41:46.858052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:41:46.858052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:41:46.858052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:41:46.858052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:41:46.858052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:41:46.858052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:41:46.858052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:41:46.858052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 622/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Aug 30 14:41:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:46.862726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174b35f-e58e-4ea7-8637-32ed953a8613 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:46.909023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:46.959383 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174b35f-e58e-4ea7-8637-32ed953a8613 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:46.992358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caca1c-2fa0-4d0c-a398-7c3055dac0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:46.992945 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caca1c-2fa0-4d0c-a398-7c3055dac0b3 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:46.993222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caca1c-2fa0-4d0c-a398-7c3055dac0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:46.993490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caca1c-2fa0-4d0c-a398-7c3055dac0b3 None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:46.994198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9caca1c-2fa0-4d0c-a398-7c3055dac0b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:46.994497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9caca1c-2fa0-4d0c-a398-7c3055dac0b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:46.995881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9caca1c-2fa0-4d0c-a398-7c3055dac0b3 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:47.004047 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 614/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Aug 30 14:41:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:47.009634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cad35-dd24-4f22-8c8f-02d045119562 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:47.092705 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:47.107856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cad35-dd24-4f22-8c8f-02d045119562 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:47.140593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb008791-0aca-439d-a1b7-2d0429cc46ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:47.141469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb008791-0aca-439d-a1b7-2d0429cc46ef None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:47.141740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb008791-0aca-439d-a1b7-2d0429cc46ef None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:47.142560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb008791-0aca-439d-a1b7-2d0429cc46ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:47.153051 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb008791-0aca-439d-a1b7-2d0429cc46ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:47.154528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb008791-0aca-439d-a1b7-2d0429cc46ef None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:47.168757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-fb008791-0aca-439d-a1b7-2d0429cc46ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:47.168757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:47.168757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:47.168757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:47.168757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:47.168757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:47.168757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:47.168757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:47.168757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:47.168757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:47.168757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:47.168757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:47.168757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:41:47.168757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:41:47.168757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:41:47.168757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:47.169888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:41:47.171127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:41:47.171127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:47.171127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:41:47.171127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 623/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:47.175768 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caca1c-2fa0-4d0c-a398-7c3055dac0b3 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:47.276264 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caca1c-2fa0-4d0c-a398-7c3055dac0b3 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:47.284974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:47.308216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55557df8-7ece-4b20-98db-a30faf51a814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:47.308808 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55557df8-7ece-4b20-98db-a30faf51a814 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:47.309083 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55557df8-7ece-4b20-98db-a30faf51a814 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:47.309352 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55557df8-7ece-4b20-98db-a30faf51a814 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:47.309985 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55557df8-7ece-4b20-98db-a30faf51a814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:47.310286 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55557df8-7ece-4b20-98db-a30faf51a814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:47.311703 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55557df8-7ece-4b20-98db-a30faf51a814 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:47.323510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 615/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:47.330193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb008791-0aca-439d-a1b7-2d0429cc46ef None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:47.430320 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb008791-0aca-439d-a1b7-2d0429cc46ef None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:47.463038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbbc08-12df-4a89-a816-45262f3c5ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:47.463701 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbbc08-12df-4a89-a816-45262f3c5ce6 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:47.463976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbbc08-12df-4a89-a816-45262f3c5ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:47.464241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbbc08-12df-4a89-a816-45262f3c5ce6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:47.466819 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:47.471726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dbbc08-12df-4a89-a816-45262f3c5ce6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:47.473149 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dbbc08-12df-4a89-a816-45262f3c5ce6 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:47.483873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-b8dbbc08-12df-4a89-a816-45262f3c5ce6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:41:47.483873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:47.483873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:47.483873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:47.483873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:47.483873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:47.483873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:47.483873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:47.483873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:47.483873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:47.483873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:47.483873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:47.483873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:41:47.483873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:41:47.483873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:41:47.483873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:47.484960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:47.486444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:47.486444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:47.486444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:41:47.486444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:41:47.486444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:41:47.486444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:41:47.486444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:41:47.486444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:41:47.486444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:41:47.486444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:41:47.486444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 624/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:47.491520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55557df8-7ece-4b20-98db-a30faf51a814 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:47.590090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55557df8-7ece-4b20-98db-a30faf51a814 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:47.624797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8c2f0-dec1-4f7c-ba87-5a8fdea2046c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:47.625398 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8c2f0-dec1-4f7c-ba87-5a8fdea2046c None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:47.625675 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8c2f0-dec1-4f7c-ba87-5a8fdea2046c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:47.625946 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8c2f0-dec1-4f7c-ba87-5a8fdea2046c None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:47.627347 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8c2f0-dec1-4f7c-ba87-5a8fdea2046c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:47.627651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8c2f0-dec1-4f7c-ba87-5a8fdea2046c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:47.629064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8c2f0-dec1-4f7c-ba87-5a8fdea2046c None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:47.647723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 616/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Aug 30 14:41:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:47.654130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbbc08-12df-4a89-a816-45262f3c5ce6 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:47.755114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbbc08-12df-4a89-a816-45262f3c5ce6 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:47.788603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac04081-4704-4f83-94ed-d1473bd2db88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:47.789245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac04081-4704-4f83-94ed-d1473bd2db88 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:47.789519 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac04081-4704-4f83-94ed-d1473bd2db88 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:47.789789 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac04081-4704-4f83-94ed-d1473bd2db88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:47.848803 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:47.855127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac04081-4704-4f83-94ed-d1473bd2db88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:47.856576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac04081-4704-4f83-94ed-d1473bd2db88 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:47.953045 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 625/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:41:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:41:47.958890 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8c2f0-dec1-4f7c-ba87-5a8fdea2046c None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:47.986636 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:48.058829 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8c2f0-dec1-4f7c-ba87-5a8fdea2046c None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:48.092336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a193aab-dd63-4764-9253-4117d87d0503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:48.092953 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193aab-dd63-4764-9253-4117d87d0503 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:48.093230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193aab-dd63-4764-9253-4117d87d0503 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:48.093526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193aab-dd63-4764-9253-4117d87d0503 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:48.104293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a193aab-dd63-4764-9253-4117d87d0503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:48.105731 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a193aab-dd63-4764-9253-4117d87d0503 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:48.120761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-8a193aab-dd63-4764-9253-4117d87d0503 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:48.120761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:48.120761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:48.120761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:48.120761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:48.120761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:48.120761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:48.120761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:48.120761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:48.120761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:48.120761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:48.120761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:48.120761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:41:48.120761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:41:48.120761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:41:48.120761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:48.130244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:41:48.131353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:41:48.131353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:48.131353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:41:48.131353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 617/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:41:48.131666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac04081-4704-4f83-94ed-d1473bd2db88 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:48.219415 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:48.225837 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac04081-4704-4f83-94ed-d1473bd2db88 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:48.259310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ea4cc-8c81-471c-8d5a-f701191dd0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:48.259922 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ea4cc-8c81-471c-8d5a-f701191dd0c0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:48.260202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ea4cc-8c81-471c-8d5a-f701191dd0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:48.260472 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ea4cc-8c81-471c-8d5a-f701191dd0c0 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:48.261111 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ea4cc-8c81-471c-8d5a-f701191dd0c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:48.261413 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ea4cc-8c81-471c-8d5a-f701191dd0c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:48.262781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ea4cc-8c81-471c-8d5a-f701191dd0c0 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:48.274694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 626/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:48.280353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a193aab-dd63-4764-9253-4117d87d0503 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:48.357861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:48.378551 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a193aab-dd63-4764-9253-4117d87d0503 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:48.411566 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cadde-ee5f-4808-940a-5308c32d32d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:48.412181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cadde-ee5f-4808-940a-5308c32d32d3 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:48.412461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cadde-ee5f-4808-940a-5308c32d32d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:48.412733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cadde-ee5f-4808-940a-5308c32d32d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:48.419160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cadde-ee5f-4808-940a-5308c32d32d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:48.420587 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cadde-ee5f-4808-940a-5308c32d32d3 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:48.432118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-a86cadde-ee5f-4808-940a-5308c32d32d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:41:48.432118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:48.432118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:48.432118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:48.432118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:48.432118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:48.432118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:48.432118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:48.432118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:48.432118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:48.432118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:48.432118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:48.432118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:41:48.432118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:41:48.432118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:48.433248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:48.434283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:41:48.434283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:41:48.434283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:41:48.434283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:41:48.434283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:41:48.434283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:41:48.434283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:41:48.434283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:41:48.434283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 618/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:41:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:48.439515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ea4cc-8c81-471c-8d5a-f701191dd0c0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:48.538367 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ea4cc-8c81-471c-8d5a-f701191dd0c0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:48.571301 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fa0e6-7193-4d0f-a8e8-5485318e0bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:48.571994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fa0e6-7193-4d0f-a8e8-5485318e0bd4 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:48.572276 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fa0e6-7193-4d0f-a8e8-5485318e0bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:48.572545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fa0e6-7193-4d0f-a8e8-5485318e0bd4 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:48.573230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0fa0e6-7193-4d0f-a8e8-5485318e0bd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:48.573532 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0fa0e6-7193-4d0f-a8e8-5485318e0bd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:48.574960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0fa0e6-7193-4d0f-a8e8-5485318e0bd4 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:48.587337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 627/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:41:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:48.593579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cadde-ee5f-4808-940a-5308c32d32d3 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:48.692863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cadde-ee5f-4808-940a-5308c32d32d3 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:48.725995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f891e30-f46e-4fa6-8290-8c3473399b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:48.729667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f891e30-f46e-4fa6-8290-8c3473399b8b None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:48.729667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f891e30-f46e-4fa6-8290-8c3473399b8b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:48.729667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f891e30-f46e-4fa6-8290-8c3473399b8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:48.729667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f891e30-f46e-4fa6-8290-8c3473399b8b None admin] RBAC: Adding query filter params (role.id=244ad93ecd1949c9a6e8f74b135b43b4, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=f4f11655cd51438e9652869f74ffa975) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:48.729667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f891e30-f46e-4fa6-8290-8c3473399b8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:48.729667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f891e30-f46e-4fa6-8290-8c3473399b8b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:48.735190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 619/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:41:48 2022] GET /identity/v3/role_assignments?user.id=f4f11655cd51438e9652869f74ffa975&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=244ad93ecd1949c9a6e8f74b135b43b4 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:41:50.680077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b50b6e-bf64-4cdc-8848-2fc6bcce258e None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:50.680619 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b50b6e-bf64-4cdc-8848-2fc6bcce258e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:50.680619 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b50b6e-bf64-4cdc-8848-2fc6bcce258e None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:50.681925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 628/1247] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:41:50.687026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b2bdf-af71-487c-a5bd-2a69ff83e364 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:50.696793 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b2bdf-af71-487c-a5bd-2a69ff83e364 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:50.696793 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b2bdf-af71-487c-a5bd-2a69ff83e364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:50.696793 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:50.706494 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e42b2bdf-af71-487c-a5bd-2a69ff83e364 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:41:50.707777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e42b2bdf-af71-487c-a5bd-2a69ff83e364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:41:50.772676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 620/1248] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:41:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:41:51.038899 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b50b6e-bf64-4cdc-8848-2fc6bcce258e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:51.057637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:51.138041 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b50b6e-bf64-4cdc-8848-2fc6bcce258e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:51.171454 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c45ccd-a3fd-456f-8786-9197070f43d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:51.172064 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c45ccd-a3fd-456f-8786-9197070f43d0 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:51.172394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c45ccd-a3fd-456f-8786-9197070f43d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:51.172696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c45ccd-a3fd-456f-8786-9197070f43d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:51.184634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c45ccd-a3fd-456f-8786-9197070f43d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:41:51.195363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-a8c45ccd-a3fd-456f-8786-9197070f43d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:41:51.195363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:51.195363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:51.195363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:51.195363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:51.195363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:51.195363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:51.195363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:51.195363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:51.195363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:51.195363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:51.195363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:51.195363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:41:51.195363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:41:51.195363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:41:51.195363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:41:51.196559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:41:51.196559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:41:51.196559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:41:51.196559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:41:51.196559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:51.196559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:51.196559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:51.196559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:51.196559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:51.196559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:51.196559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:51.196559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:51.196559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:51.196559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:51.196559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:51.196559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:51.197610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:51.197610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:51.197610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:51.197610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:51.197610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:41:51.197610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:41:51.197610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:41:51.197610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:41:51.197610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:41:51.197610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:41:51.197610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:41:51.197610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:41:51.197610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 629/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:41:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:51.199600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b2bdf-af71-487c-a5bd-2a69ff83e364 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:51.315340 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b2bdf-af71-487c-a5bd-2a69ff83e364 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:51.344746 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:51.349950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108945c5-5a7e-49ca-a9c0-e8372e3c190a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:51.350544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108945c5-5a7e-49ca-a9c0-e8372e3c190a None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:51.350817 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108945c5-5a7e-49ca-a9c0-e8372e3c190a None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:51.351084 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108945c5-5a7e-49ca-a9c0-e8372e3c190a None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:51.351765 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108945c5-5a7e-49ca-a9c0-e8372e3c190a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:51.352063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108945c5-5a7e-49ca-a9c0-e8372e3c190a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:51.353419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108945c5-5a7e-49ca-a9c0-e8372e3c190a None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:51.361285 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 621/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:41:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:51.367125 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c45ccd-a3fd-456f-8786-9197070f43d0 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:51.443270 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:51.466002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c45ccd-a3fd-456f-8786-9197070f43d0 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:51.498132 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525ed87-f275-4c4d-8c82-86dbed0723ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:51.498750 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525ed87-f275-4c4d-8c82-86dbed0723ae None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:51.499028 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525ed87-f275-4c4d-8c82-86dbed0723ae None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:51.499370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525ed87-f275-4c4d-8c82-86dbed0723ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:51.514216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525ed87-f275-4c4d-8c82-86dbed0723ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:51.515694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525ed87-f275-4c4d-8c82-86dbed0723ae None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:51.530423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-c525ed87-f275-4c4d-8c82-86dbed0723ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:51.530423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:51.530423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:51.530423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:51.530423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:51.530423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:51.530423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:51.530423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:51.530423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:51.530423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:51.530423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:51.530423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:51.530423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:41:51.530423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:41:51.530423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:41:51.530423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:51.531635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:41:51.532876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:41:51.532876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:51.532876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:41:51.532876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 630/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:41:51.537652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108945c5-5a7e-49ca-a9c0-e8372e3c190a None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:51.637077 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108945c5-5a7e-49ca-a9c0-e8372e3c190a None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:51.670824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a2884-32a1-458a-b429-ed3affa0cccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:51.671510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a2884-32a1-458a-b429-ed3affa0cccf None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:51.671797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a2884-32a1-458a-b429-ed3affa0cccf None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:51.672071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a2884-32a1-458a-b429-ed3affa0cccf None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:51.672708 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a2884-32a1-458a-b429-ed3affa0cccf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:51.673036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a2884-32a1-458a-b429-ed3affa0cccf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:51.674390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a2884-32a1-458a-b429-ed3affa0cccf None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:51.685884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 622/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:51.692188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525ed87-f275-4c4d-8c82-86dbed0723ae None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:51.790545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525ed87-f275-4c4d-8c82-86dbed0723ae None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:51.815817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:51.823213 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9ab77-c5b6-454d-b896-8479eedd5b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:51.823900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9ab77-c5b6-454d-b896-8479eedd5b5f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:51.824108 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9ab77-c5b6-454d-b896-8479eedd5b5f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:51.824377 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9ab77-c5b6-454d-b896-8479eedd5b5f None admin] PATH_INFO: `/v3/users/placement` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:51.834627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9ab77-c5b6-454d-b896-8479eedd5b5f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:51.836112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9ab77-c5b6-454d-b896-8479eedd5b5f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:51.847840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-6be9ab77-c5b6-454d-b896-8479eedd5b5f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:41:51.847840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:51.847840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:51.847840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:51.847840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:51.847840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:51.847840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:51.847840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:51.847840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:51.847840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:51.847840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:51.847840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:51.847840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:41:51.847840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:41:51.847840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:41:51.847840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:51.848977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:51.850140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:51.850140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:51.850140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:41:51.850140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:41:51.850140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:41:51.850140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:41:51.850140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:41:51.850140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:41:51.850140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:41:51.850140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:41:51.850140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 631/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:41:51.854781 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a2884-32a1-458a-b429-ed3affa0cccf None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:51.909271 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:51.951810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a2884-32a1-458a-b429-ed3affa0cccf None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:51.983337 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97009961-ac2b-44bf-9cb8-e68347af157b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:51.983923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97009961-ac2b-44bf-9cb8-e68347af157b None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:51.984201 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97009961-ac2b-44bf-9cb8-e68347af157b None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:51.984466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97009961-ac2b-44bf-9cb8-e68347af157b None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:51.985181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97009961-ac2b-44bf-9cb8-e68347af157b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:51.985981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97009961-ac2b-44bf-9cb8-e68347af157b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:51.987411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97009961-ac2b-44bf-9cb8-e68347af157b None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:52.004317 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 623/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Aug 30 14:41:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:52.009935 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9ab77-c5b6-454d-b896-8479eedd5b5f None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:52.107677 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9ab77-c5b6-454d-b896-8479eedd5b5f None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:52.140693 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e4fb2-8c65-4c9b-8fce-2430c24667f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:52.141579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e4fb2-8c65-4c9b-8fce-2430c24667f7 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:52.141857 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e4fb2-8c65-4c9b-8fce-2430c24667f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:52.142126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e4fb2-8c65-4c9b-8fce-2430c24667f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:52.195041 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:52.206680 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e4fb2-8c65-4c9b-8fce-2430c24667f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:52.208162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e4fb2-8c65-4c9b-8fce-2430c24667f7 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:52.305635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 632/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:41:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 14:41:52.311537 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97009961-ac2b-44bf-9cb8-e68347af157b None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:52.410039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97009961-ac2b-44bf-9cb8-e68347af157b None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:52.423936 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:52.444256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfec162-77d4-4111-89ce-c6ddbb4b9901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:52.444988 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfec162-77d4-4111-89ce-c6ddbb4b9901 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:52.445314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfec162-77d4-4111-89ce-c6ddbb4b9901 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:52.445540 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfec162-77d4-4111-89ce-c6ddbb4b9901 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:52.455381 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfec162-77d4-4111-89ce-c6ddbb4b9901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:52.456834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfec162-77d4-4111-89ce-c6ddbb4b9901 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:52.472202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-bcfec162-77d4-4111-89ce-c6ddbb4b9901 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:52.472202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:52.472202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:52.472202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:52.472202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:52.472202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:52.472202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:52.472202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:52.472202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:52.472202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:52.472202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:52.472202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:52.472202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:41:52.472202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:41:52.472202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:41:52.472202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:52.473331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:41:52.474412 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:41:52.474412 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:52.474412 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:41:52.474412 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 624/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:52.479030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e4fb2-8c65-4c9b-8fce-2430c24667f7 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:52.565496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:52.578937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e4fb2-8c65-4c9b-8fce-2430c24667f7 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:52.612134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88024b90-48f8-490d-94a5-cc7cf9aae0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:52.612737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88024b90-48f8-490d-94a5-cc7cf9aae0e1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:52.613014 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88024b90-48f8-490d-94a5-cc7cf9aae0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:52.613283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88024b90-48f8-490d-94a5-cc7cf9aae0e1 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:52.613917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88024b90-48f8-490d-94a5-cc7cf9aae0e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:52.614216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88024b90-48f8-490d-94a5-cc7cf9aae0e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:52.615622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88024b90-48f8-490d-94a5-cc7cf9aae0e1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:52.626666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 633/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:52.632212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfec162-77d4-4111-89ce-c6ddbb4b9901 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:52.731426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfec162-77d4-4111-89ce-c6ddbb4b9901 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:52.764925 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536422c3-32b9-4957-9476-96db28a9226f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:52.765546 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536422c3-32b9-4957-9476-96db28a9226f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:52.766067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536422c3-32b9-4957-9476-96db28a9226f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:52.766334 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536422c3-32b9-4957-9476-96db28a9226f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:52.773334 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536422c3-32b9-4957-9476-96db28a9226f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:52.774773 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536422c3-32b9-4957-9476-96db28a9226f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:52.786074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-536422c3-32b9-4957-9476-96db28a9226f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:41:52.786074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:52.786074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:52.786074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:52.786074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:52.786074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:52.786074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:52.786074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:52.786074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:52.786074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:52.786074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:52.786074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:52.786074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:41:52.786074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:41:52.786074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:52.787253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:52.788376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:41:52.788376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:41:52.788376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:41:52.788376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:41:52.788376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:41:52.788376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:41:52.788376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:41:52.788376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:41:52.788376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 625/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:41:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:52.792974 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88024b90-48f8-490d-94a5-cc7cf9aae0e1 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:52.891944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88024b90-48f8-490d-94a5-cc7cf9aae0e1 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:52.924236 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d386d-fae8-485c-8b61-ccc01b06bfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:52.924824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d386d-fae8-485c-8b61-ccc01b06bfe9 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:52.925105 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d386d-fae8-485c-8b61-ccc01b06bfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:52.925372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d386d-fae8-485c-8b61-ccc01b06bfe9 None admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:52.926102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d386d-fae8-485c-8b61-ccc01b06bfe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:52.926398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d386d-fae8-485c-8b61-ccc01b06bfe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:52.927839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d386d-fae8-485c-8b61-ccc01b06bfe9 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:52.940090 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 634/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:41:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:52.946332 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536422c3-32b9-4957-9476-96db28a9226f None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:52.971393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:53.042520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536422c3-32b9-4957-9476-96db28a9226f None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:53.075958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703432ff-5957-4863-9f70-c5ea386d44c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:53.076600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703432ff-5957-4863-9f70-c5ea386d44c3 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:53.076876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703432ff-5957-4863-9f70-c5ea386d44c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:53.077146 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703432ff-5957-4863-9f70-c5ea386d44c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:53.077868 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703432ff-5957-4863-9f70-c5ea386d44c3 None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=f4f11655cd51438e9652869f74ffa975) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:53.078173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703432ff-5957-4863-9f70-c5ea386d44c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:53.079654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703432ff-5957-4863-9f70-c5ea386d44c3 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:53.084932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 626/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:41:52 2022] GET /identity/v3/role_assignments?user.id=f4f11655cd51438e9652869f74ffa975&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:54.950813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df406447-d73c-48c8-ac79-d5237607a79b None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:54.951402 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df406447-d73c-48c8-ac79-d5237607a79b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:54.951402 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df406447-d73c-48c8-ac79-d5237607a79b None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:54.952729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 635/1261] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:41:54.957880 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53553c6-92db-442b-a981-8a5f00205e24 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:54.958258 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53553c6-92db-442b-a981-8a5f00205e24 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:54.958440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53553c6-92db-442b-a981-8a5f00205e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:54.976560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-b53553c6-92db-442b-a981-8a5f00205e24 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:41:54.977758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b53553c6-92db-442b-a981-8a5f00205e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:41:55.044506 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 627/1262] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:41:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:41:55.318584 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df406447-d73c-48c8-ac79-d5237607a79b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:55.330254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:55.420761 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df406447-d73c-48c8-ac79-d5237607a79b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:55.453807 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978f3c9-8701-446e-864b-e9dd4f13f2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:55.454977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978f3c9-8701-446e-864b-e9dd4f13f2a6 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:55.455286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978f3c9-8701-446e-864b-e9dd4f13f2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:55.455558 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978f3c9-8701-446e-864b-e9dd4f13f2a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:55.465306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978f3c9-8701-446e-864b-e9dd4f13f2a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:41:55.470094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-8978f3c9-8701-446e-864b-e9dd4f13f2a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:41:55.470094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:55.470094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:55.470094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:55.470094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:55.470094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:55.470094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:55.470094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:55.470094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:55.470094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:55.470094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:55.470094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:55.470094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:41:55.470094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:41:55.470094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:41:55.470094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:41:55.471231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:41:55.471231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:41:55.471231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:41:55.471231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:41:55.471231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:55.471231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:55.471231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:55.471231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:55.471231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:55.471231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:55.471231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:55.471231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:55.471231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:55.471231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:55.471231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:55.471231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:55.472085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:55.472085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:55.472085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:55.472085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:55.472085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:41:55.472085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:41:55.472085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:41:55.472085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:41:55.472085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:41:55.472085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:41:55.472085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:41:55.472085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:41:55.472085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 636/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:41:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:55.477295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53553c6-92db-442b-a981-8a5f00205e24 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:55.494848 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:55.573304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53553c6-92db-442b-a981-8a5f00205e24 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:55.607348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32fa0f-81c4-479b-94d4-9a99f7d021ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:55.607968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32fa0f-81c4-479b-94d4-9a99f7d021ed None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:55.608242 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32fa0f-81c4-479b-94d4-9a99f7d021ed None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:55.608507 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32fa0f-81c4-479b-94d4-9a99f7d021ed None admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:55.609154 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32fa0f-81c4-479b-94d4-9a99f7d021ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:55.609452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32fa0f-81c4-479b-94d4-9a99f7d021ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:55.611331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32fa0f-81c4-479b-94d4-9a99f7d021ed None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:55.619152 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 628/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:41:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:55.624988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978f3c9-8701-446e-864b-e9dd4f13f2a6 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:55.690791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:55.723696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978f3c9-8701-446e-864b-e9dd4f13f2a6 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:55.759594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81172081-8696-4b6d-a73c-51ec6f607068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:55.760212 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81172081-8696-4b6d-a73c-51ec6f607068 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:55.760491 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81172081-8696-4b6d-a73c-51ec6f607068 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:55.760764 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81172081-8696-4b6d-a73c-51ec6f607068 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:55.770708 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81172081-8696-4b6d-a73c-51ec6f607068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:55.772189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81172081-8696-4b6d-a73c-51ec6f607068 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:55.787286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-81172081-8696-4b6d-a73c-51ec6f607068 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:55.787286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:55.787286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:55.787286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:55.787286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:55.787286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:55.787286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:55.787286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:55.787286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:55.787286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:55.787286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:55.787286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:55.787286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:41:55.787286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:41:55.787286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:41:55.787286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:55.788448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:41:55.789389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:41:55.789389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:55.789389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:41:55.789389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 637/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:55.794412 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32fa0f-81c4-479b-94d4-9a99f7d021ed None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:55.873721 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:55.892475 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32fa0f-81c4-479b-94d4-9a99f7d021ed None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:55.925614 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9460f2-fb94-448b-9909-499a71546451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:55.926209 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9460f2-fb94-448b-9909-499a71546451 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:55.926488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9460f2-fb94-448b-9909-499a71546451 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:55.926760 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9460f2-fb94-448b-9909-499a71546451 None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:55.927681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9460f2-fb94-448b-9909-499a71546451 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:55.928092 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9460f2-fb94-448b-9909-499a71546451 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:55.929445 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9460f2-fb94-448b-9909-499a71546451 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:55.940924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 629/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:55.946646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81172081-8696-4b6d-a73c-51ec6f607068 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:56.045672 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81172081-8696-4b6d-a73c-51ec6f607068 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:56.065068 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:56.079005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fede1-10d7-4126-8c51-5b42a3e77694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:56.079662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fede1-10d7-4126-8c51-5b42a3e77694 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:56.079945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fede1-10d7-4126-8c51-5b42a3e77694 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:56.080223 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fede1-10d7-4126-8c51-5b42a3e77694 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:56.087578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fede1-10d7-4126-8c51-5b42a3e77694 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:56.089012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fede1-10d7-4126-8c51-5b42a3e77694 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:56.100202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-d44fede1-10d7-4126-8c51-5b42a3e77694 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:41:56.100202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:56.100202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:56.100202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:56.100202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:56.100202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:56.100202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:56.100202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:56.100202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:56.100202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:56.100202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:56.100202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:56.100202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:41:56.100202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:41:56.100202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:41:56.100202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:56.101455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:56.102487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:56.102487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:56.102487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:41:56.102487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:41:56.102487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:41:56.102487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:41:56.102487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:41:56.102487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:41:56.102487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:41:56.102487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:41:56.102487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 638/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:56.107202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9460f2-fb94-448b-9909-499a71546451 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:56.206358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9460f2-fb94-448b-9909-499a71546451 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:56.239615 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7ec10-98d4-4660-be91-ce5bcca09405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:56.240213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7ec10-98d4-4660-be91-ce5bcca09405 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:56.240486 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7ec10-98d4-4660-be91-ce5bcca09405 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:56.240752 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7ec10-98d4-4660-be91-ce5bcca09405 None admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:56.241473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff7ec10-98d4-4660-be91-ce5bcca09405 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:56.241776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff7ec10-98d4-4660-be91-ce5bcca09405 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:56.243229 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff7ec10-98d4-4660-be91-ce5bcca09405 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:56.252308 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:56.260429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 630/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Aug 30 14:41:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:56.266580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fede1-10d7-4126-8c51-5b42a3e77694 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:56.365154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fede1-10d7-4126-8c51-5b42a3e77694 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:56.399216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3432fc8-b95f-4454-8921-77f21cba6bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:56.399901 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3432fc8-b95f-4454-8921-77f21cba6bbe None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:56.400184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3432fc8-b95f-4454-8921-77f21cba6bbe None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:56.400456 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3432fc8-b95f-4454-8921-77f21cba6bbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:56.411253 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3432fc8-b95f-4454-8921-77f21cba6bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:56.412689 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3432fc8-b95f-4454-8921-77f21cba6bbe None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:56.426306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-b3432fc8-b95f-4454-8921-77f21cba6bbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:56.426306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:56.426306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:56.426306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:56.426306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:56.426306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:56.426306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:56.426306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:56.426306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:56.426306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:56.426306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:56.426306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:56.426306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:41:56.426306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:41:56.426306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:41:56.426306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:56.427404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:41:56.428473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:41:56.428473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:56.428473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:41:56.428473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 639/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:56.433359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7ec10-98d4-4660-be91-ce5bcca09405 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:56.531862 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7ec10-98d4-4660-be91-ce5bcca09405 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:56.565708 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3a4d2-4728-4f93-838f-1259dff9cfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:56.566308 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3a4d2-4728-4f93-838f-1259dff9cfea None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:56.566587 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3a4d2-4728-4f93-838f-1259dff9cfea None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:56.566859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3a4d2-4728-4f93-838f-1259dff9cfea None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:56.567616 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3a4d2-4728-4f93-838f-1259dff9cfea None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:56.567920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3a4d2-4728-4f93-838f-1259dff9cfea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:56.569282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3a4d2-4728-4f93-838f-1259dff9cfea None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:56.580800 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 631/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:56.586542 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3432fc8-b95f-4454-8921-77f21cba6bbe None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:56.619210 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:56.685735 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3432fc8-b95f-4454-8921-77f21cba6bbe None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:56.718733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059ae7f-6a9f-4c5f-8a23-af051f42e511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:56.719385 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059ae7f-6a9f-4c5f-8a23-af051f42e511 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:56.720306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059ae7f-6a9f-4c5f-8a23-af051f42e511 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:56.720574 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059ae7f-6a9f-4c5f-8a23-af051f42e511 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:56.726996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059ae7f-6a9f-4c5f-8a23-af051f42e511 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:56.728456 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059ae7f-6a9f-4c5f-8a23-af051f42e511 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:56.739925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-d059ae7f-6a9f-4c5f-8a23-af051f42e511 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:41:56.739925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:56.739925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:56.739925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:56.739925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:56.739925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:56.739925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:56.739925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:56.739925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:56.739925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:56.739925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:56.739925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:56.739925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:41:56.739925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:41:56.739925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:56.740954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:56.741873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:41:56.741873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:41:56.741873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:41:56.741873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:41:56.741873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:41:56.741873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:41:56.741873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:41:56.741873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:41:56.741873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 640/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:41:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:56.746814 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3a4d2-4728-4f93-838f-1259dff9cfea None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:56.803673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:56.845851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3a4d2-4728-4f93-838f-1259dff9cfea None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:56.878125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ab044-dd9c-4f5a-a375-2993144f0122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:56.878716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ab044-dd9c-4f5a-a375-2993144f0122 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:56.879012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ab044-dd9c-4f5a-a375-2993144f0122 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:56.879292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ab044-dd9c-4f5a-a375-2993144f0122 None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:56.879973 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ab044-dd9c-4f5a-a375-2993144f0122 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:56.880274 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ab044-dd9c-4f5a-a375-2993144f0122 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:56.881685 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ab044-dd9c-4f5a-a375-2993144f0122 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:56.894055 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 632/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:41:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:41:56.900280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059ae7f-6a9f-4c5f-8a23-af051f42e511 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:56.990144 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:56.998616 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059ae7f-6a9f-4c5f-8a23-af051f42e511 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:57.032418 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605877b0-c84f-4b2e-9234-93985b32f368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:57.033061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605877b0-c84f-4b2e-9234-93985b32f368 None admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:57.033340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605877b0-c84f-4b2e-9234-93985b32f368 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:57.033610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605877b0-c84f-4b2e-9234-93985b32f368 None admin] PATH_INFO: `/v3/projects/1147574d59c7404baf76a24149684f6e/users/f4f11655cd51438e9652869f74ffa975/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:57.035499 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605877b0-c84f-4b2e-9234-93985b32f368 None admin] RBAC: Authorizing `identity:create_grant(project_id=1147574d59c7404baf76a24149684f6e, user_id=f4f11655cd51438e9652869f74ffa975, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:57.036925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605877b0-c84f-4b2e-9234-93985b32f368 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:57.140055 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 641/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Aug 30 14:41:56 2022] PUT /identity/v3/projects/1147574d59c7404baf76a24149684f6e/users/f4f11655cd51438e9652869f74ffa975/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:41:59.023368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca9e4a-2c25-4517-9822-7e125b6d6d05 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:59.023898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca9e4a-2c25-4517-9822-7e125b6d6d05 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:59.023991 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca9e4a-2c25-4517-9822-7e125b6d6d05 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:59.025236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 633/1274] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:41:59.030473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4230f08-e06f-4e40-ba9b-dd68183ba9e4 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:59.030907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4230f08-e06f-4e40-ba9b-dd68183ba9e4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:59.031007 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4230f08-e06f-4e40-ba9b-dd68183ba9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:59.049900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c4230f08-e06f-4e40-ba9b-dd68183ba9e4 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:41:59.051177 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c4230f08-e06f-4e40-ba9b-dd68183ba9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:41:59.115369 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 642/1275] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:41:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:41:59.379682 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca9e4a-2c25-4517-9822-7e125b6d6d05 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:59.480953 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca9e4a-2c25-4517-9822-7e125b6d6d05 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:59.497908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:59.514410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd38828-68a5-4484-ba5b-9d779d66d623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:59.515029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd38828-68a5-4484-ba5b-9d779d66d623 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:59.515701 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd38828-68a5-4484-ba5b-9d779d66d623 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:59.515976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd38828-68a5-4484-ba5b-9d779d66d623 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:59.526531 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd38828-68a5-4484-ba5b-9d779d66d623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:41:59.531272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-6bd38828-68a5-4484-ba5b-9d779d66d623 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:41:59.531272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:59.531272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:59.531272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:59.531272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:59.531272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:59.531272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:59.531272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:59.531272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:59.531272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:59.531272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:59.531272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:59.531272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:41:59.531272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:41:59.531272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:41:59.531272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:41:59.532923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:41:59.532923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:41:59.532923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:41:59.532923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:41:59.532923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:59.532923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:59.532923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:59.532923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:59.532923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:59.532923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:59.532923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:59.532923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:59.532923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:59.532923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:59.532923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:59.532923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:59.534320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:59.534320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:59.534320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:59.534320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:59.534320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:41:59.534320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:41:59.534320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:41:59.534320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:41:59.534320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:41:59.534320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:41:59.534320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:41:59.534320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:41:59.534320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 634/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Aug 30 14:41:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:41:59.539046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4230f08-e06f-4e40-ba9b-dd68183ba9e4 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:59.616963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:41:59.634169 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4230f08-e06f-4e40-ba9b-dd68183ba9e4 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:59.667910 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252fbdd-7ebd-4083-af57-2b516d9c6651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:59.668502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252fbdd-7ebd-4083-af57-2b516d9c6651 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:59.668776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252fbdd-7ebd-4083-af57-2b516d9c6651 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:59.669044 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252fbdd-7ebd-4083-af57-2b516d9c6651 None admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:59.669701 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252fbdd-7ebd-4083-af57-2b516d9c6651 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:59.670026 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252fbdd-7ebd-4083-af57-2b516d9c6651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:59.671439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252fbdd-7ebd-4083-af57-2b516d9c6651 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:59.679133 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 643/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Aug 30 14:41:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:41:59.684758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd38828-68a5-4484-ba5b-9d779d66d623 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:59.785103 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd38828-68a5-4484-ba5b-9d779d66d623 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:59.818436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563958cf-3625-4c48-8992-f3dac776ab7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:59.819040 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563958cf-3625-4c48-8992-f3dac776ab7d None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:59.819340 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563958cf-3625-4c48-8992-f3dac776ab7d None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:59.819605 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563958cf-3625-4c48-8992-f3dac776ab7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:59.829801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563958cf-3625-4c48-8992-f3dac776ab7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:59.833638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563958cf-3625-4c48-8992-f3dac776ab7d None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:41:59.845743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-563958cf-3625-4c48-8992-f3dac776ab7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:59.845743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:41:59.845743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:41:59.845743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:41:59.845743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:41:59.845743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:41:59.845743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:41:59.845743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:41:59.845743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:41:59.845743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:41:59.845743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:41:59.845743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:41:59.845743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:41:59.845743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:41:59.845743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:41:59.845743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:41:59.846845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:41:59.847982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:41:59.847982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:41:59.847982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:41:59.847982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 635/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:41:59.852717 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252fbdd-7ebd-4083-af57-2b516d9c6651 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:41:59.957986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252fbdd-7ebd-4083-af57-2b516d9c6651 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:41:59.989563 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42895cd-6c15-4eab-974a-f0bc2b85b6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:41:59.990154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42895cd-6c15-4eab-974a-f0bc2b85b6b1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:41:59.990428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42895cd-6c15-4eab-974a-f0bc2b85b6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:41:59.990695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42895cd-6c15-4eab-974a-f0bc2b85b6b1 None admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:41:59.991351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42895cd-6c15-4eab-974a-f0bc2b85b6b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:41:59.991650 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42895cd-6c15-4eab-974a-f0bc2b85b6b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:41:59.993010 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42895cd-6c15-4eab-974a-f0bc2b85b6b1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:00.005767 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 644/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:00.011293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563958cf-3625-4c48-8992-f3dac776ab7d None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:00.042332 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:00.108828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563958cf-3625-4c48-8992-f3dac776ab7d None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:00.142611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db1683-ca97-4c43-bddf-52ca11717b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:00.143279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db1683-ca97-4c43-bddf-52ca11717b32 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:00.143560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db1683-ca97-4c43-bddf-52ca11717b32 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:00.143830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db1683-ca97-4c43-bddf-52ca11717b32 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:00.150720 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db1683-ca97-4c43-bddf-52ca11717b32 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:00.152180 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db1683-ca97-4c43-bddf-52ca11717b32 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:00.165567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-55db1683-ca97-4c43-bddf-52ca11717b32 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:42:00.165567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:42:00.165567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:42:00.165567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:42:00.165567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:42:00.165567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:42:00.165567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:42:00.165567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:42:00.165567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:42:00.165567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:42:00.165567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:42:00.165567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:42:00.165567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:42:00.165567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:42:00.165567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:42:00.165567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:42:00.166835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:42:00.173110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:42:00.173110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:42:00.173110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:42:00.173110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:42:00.173110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:42:00.173110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:42:00.173110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:42:00.173110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:42:00.173110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:42:00.173110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:42:00.173110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 636/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:42:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:42:00.173830 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42895cd-6c15-4eab-974a-f0bc2b85b6b1 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:00.176949 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:00.280756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42895cd-6c15-4eab-974a-f0bc2b85b6b1 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:00.313947 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a226074-9d2c-4ff4-8785-66b51d2edaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:00.314536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a226074-9d2c-4ff4-8785-66b51d2edaa1 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:00.314811 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a226074-9d2c-4ff4-8785-66b51d2edaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:00.315077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a226074-9d2c-4ff4-8785-66b51d2edaa1 None admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:00.315825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a226074-9d2c-4ff4-8785-66b51d2edaa1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:00.316123 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a226074-9d2c-4ff4-8785-66b51d2edaa1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:00.317541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a226074-9d2c-4ff4-8785-66b51d2edaa1 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:00.334239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 645/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Aug 30 14:42:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:42:00.340069 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db1683-ca97-4c43-bddf-52ca11717b32 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:00.429387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:00.438950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db1683-ca97-4c43-bddf-52ca11717b32 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:00.471488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef4fc1-1793-4bf6-8d76-dc7c48ede98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:00.475776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef4fc1-1793-4bf6-8d76-dc7c48ede98f None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:00.475776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef4fc1-1793-4bf6-8d76-dc7c48ede98f None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:00.475776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef4fc1-1793-4bf6-8d76-dc7c48ede98f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:00.538000 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef4fc1-1793-4bf6-8d76-dc7c48ede98f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:00.539403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef4fc1-1793-4bf6-8d76-dc7c48ede98f None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:00.627392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:00.637263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 637/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Aug 30 14:42:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:42:00.643368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a226074-9d2c-4ff4-8785-66b51d2edaa1 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:00.703301 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:00.741593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a226074-9d2c-4ff4-8785-66b51d2edaa1 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:00.774605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f1d4e-b1c0-461a-a6f6-5b40b1336aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:00.775237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f1d4e-b1c0-461a-a6f6-5b40b1336aeb None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:00.775519 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f1d4e-b1c0-461a-a6f6-5b40b1336aeb None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:00.775788 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f1d4e-b1c0-461a-a6f6-5b40b1336aeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:00.786103 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f1d4e-b1c0-461a-a6f6-5b40b1336aeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:00.787606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f1d4e-b1c0-461a-a6f6-5b40b1336aeb None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:00.802335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-475f1d4e-b1c0-461a-a6f6-5b40b1336aeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:42:00.802335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:42:00.802335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:42:00.802335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:42:00.802335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:42:00.802335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:42:00.802335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:42:00.802335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:42:00.802335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:42:00.802335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:42:00.802335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:42:00.802335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:42:00.802335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:42:00.802335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:42:00.802335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:42:00.802335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:42:00.803460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:42:00.804742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:42:00.804742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:42:00.804742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:42:00.804742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 646/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:42:00.809075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef4fc1-1793-4bf6-8d76-dc7c48ede98f None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:00.906339 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef4fc1-1793-4bf6-8d76-dc7c48ede98f None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:00.940377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e856fc-f1f9-4659-84b9-e8dfa3d4623c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:00.940984 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e856fc-f1f9-4659-84b9-e8dfa3d4623c None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:00.941265 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e856fc-f1f9-4659-84b9-e8dfa3d4623c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:00.941538 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e856fc-f1f9-4659-84b9-e8dfa3d4623c None admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:00.942203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e856fc-f1f9-4659-84b9-e8dfa3d4623c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:00.942507 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e856fc-f1f9-4659-84b9-e8dfa3d4623c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:00.943919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e856fc-f1f9-4659-84b9-e8dfa3d4623c None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:00.955578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 638/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Aug 30 14:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:00.961333 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f1d4e-b1c0-461a-a6f6-5b40b1336aeb None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:01.059991 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f1d4e-b1c0-461a-a6f6-5b40b1336aeb None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:01.072966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:01.092560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa9694-e411-400b-abc7-606e73fcee96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:01.093173 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa9694-e411-400b-abc7-606e73fcee96 None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:01.093453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa9694-e411-400b-abc7-606e73fcee96 None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:01.093725 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa9694-e411-400b-abc7-606e73fcee96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:01.100718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa9694-e411-400b-abc7-606e73fcee96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:01.102162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa9694-e411-400b-abc7-606e73fcee96 None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:01.113163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application [None req-54aa9694-e411-400b-abc7-606e73fcee96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:42:01.113163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:42:01.113163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:42:01.113163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:42:01.113163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:42:01.113163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:42:01.113163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:42:01.113163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:42:01.113163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:42:01.113163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:42:01.113163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:42:01.113163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:42:01.113163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:42:01.113163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:42:01.113163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application with Lock( Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:42:01.120602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:42:01.121719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:42:01.121719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:42:01.121719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:42:01.121719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:42:01.121719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:42:01.121719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:42:01.121719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:42:01.121719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: ERROR keystone.server.flask.application Aug 30 14:42:01.121719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 647/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Aug 30 14:42:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:42:01.122330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e856fc-f1f9-4659-84b9-e8dfa3d4623c None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:01.170999 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:01.218896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e856fc-f1f9-4659-84b9-e8dfa3d4623c None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:01.252100 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313805fb-6ba3-46bb-9f8c-8532ac04eb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:01.252696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313805fb-6ba3-46bb-9f8c-8532ac04eb0c None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:01.252984 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313805fb-6ba3-46bb-9f8c-8532ac04eb0c None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:01.253261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313805fb-6ba3-46bb-9f8c-8532ac04eb0c None admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:01.254157 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313805fb-6ba3-46bb-9f8c-8532ac04eb0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:01.254624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313805fb-6ba3-46bb-9f8c-8532ac04eb0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:01.256557 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313805fb-6ba3-46bb-9f8c-8532ac04eb0c None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:01.269288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 639/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Aug 30 14:42:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:01.275442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa9694-e411-400b-abc7-606e73fcee96 None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:01.372803 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa9694-e411-400b-abc7-606e73fcee96 None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:01.406335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b7480-9326-4e20-b901-625ec04183ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:01.406925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b7480-9326-4e20-b901-625ec04183ca None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:01.407235 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b7480-9326-4e20-b901-625ec04183ca None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:01.407508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b7480-9326-4e20-b901-625ec04183ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:01.408265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b7480-9326-4e20-b901-625ec04183ca None admin] RBAC: Adding query filter params (role.id=d6822e7f42594dc4ba880ca6b9e13e0c, scope.project.id=1147574d59c7404baf76a24149684f6e, user.id=f4f11655cd51438e9652869f74ffa975) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:01.408577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b7480-9326-4e20-b901-625ec04183ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:01.410027 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b7480-9326-4e20-b901-625ec04183ca None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:01.415654 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 648/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Aug 30 14:42:01 2022] GET /identity/v3/role_assignments?user.id=f4f11655cd51438e9652869f74ffa975&scope.project.id=1147574d59c7404baf76a24149684f6e&role.id=d6822e7f42594dc4ba880ca6b9e13e0c => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:03.348505 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145d3ea-6649-48b6-a6a5-2322c305258a None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:03.349020 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145d3ea-6649-48b6-a6a5-2322c305258a None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:03.349145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145d3ea-6649-48b6-a6a5-2322c305258a None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:03.350529 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 640/1288] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:03.355800 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be56b18-d742-4904-8c5f-95f7656c5922 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:03.356184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be56b18-d742-4904-8c5f-95f7656c5922 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:03.356350 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be56b18-d742-4904-8c5f-95f7656c5922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:03.375578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3be56b18-d742-4904-8c5f-95f7656c5922 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:03.376759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3be56b18-d742-4904-8c5f-95f7656c5922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:03.408322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:03.444223 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 649/1289] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:42:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:42:03.711847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145d3ea-6649-48b6-a6a5-2322c305258a None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:03.813296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145d3ea-6649-48b6-a6a5-2322c305258a None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:03.826038 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:03.851866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f8ab2-4e98-439e-9d90-85bc3e0283e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:03.852513 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f8ab2-4e98-439e-9d90-85bc3e0283e5 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:03.852843 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f8ab2-4e98-439e-9d90-85bc3e0283e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:03.853115 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f8ab2-4e98-439e-9d90-85bc3e0283e5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:03.853713 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f8ab2-4e98-439e-9d90-85bc3e0283e5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:03.855123 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f8ab2-4e98-439e-9d90-85bc3e0283e5 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-805f8ab2-4e98-439e-9d90-85bc3e0283e5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:42:03.865460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:42:03.866803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:42:03.866803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:42:03.866803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:42:03.866803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:42:03.866803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:42:03.866803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:42:03.866803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:42:03.866803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:42:03.866803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:42:03.866803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:42:03.866803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:42:03.866803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:42:03.866803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:42:03.866803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:42:03.866803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:42:03.866803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:42:03.868036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:42:03.868036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:42:03.868036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:42:03.868036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:42:03.868036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:42:03.868036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:42:03.868036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:42:03.868036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:42:03.868036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:42:03.868036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:42:03.868036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:42:03.868036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:42:03.868036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:42:03.868036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:42:03.869134 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 641/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 30 14:42:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:42:03.872858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be56b18-d742-4904-8c5f-95f7656c5922 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:03.983566 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be56b18-d742-4904-8c5f-95f7656c5922 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:04.021526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb174eb0-34f1-40ae-a83e-92abb871c96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:04.022142 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb174eb0-34f1-40ae-a83e-92abb871c96d None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:04.022422 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb174eb0-34f1-40ae-a83e-92abb871c96d None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:04.022693 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb174eb0-34f1-40ae-a83e-92abb871c96d None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:04.023368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb174eb0-34f1-40ae-a83e-92abb871c96d None admin] RBAC: Adding query filter params (name=placement) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:04.023679 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb174eb0-34f1-40ae-a83e-92abb871c96d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:04.025112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb174eb0-34f1-40ae-a83e-92abb871c96d None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:04.030529 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 650/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Tue Aug 30 14:42:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:04.036725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f8ab2-4e98-439e-9d90-85bc3e0283e5 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:04.143555 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f8ab2-4e98-439e-9d90-85bc3e0283e5 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:04.177501 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef9d8f-03f6-4f56-8c45-f7b8ac5084e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:04.178106 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef9d8f-03f6-4f56-8c45-f7b8ac5084e5 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:04.178408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef9d8f-03f6-4f56-8c45-f7b8ac5084e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:04.178685 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef9d8f-03f6-4f56-8c45-f7b8ac5084e5 None admin] PATH_INFO: `/v3/services` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:04.179346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef9d8f-03f6-4f56-8c45-f7b8ac5084e5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:04.179650 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef9d8f-03f6-4f56-8c45-f7b8ac5084e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:04.181035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef9d8f-03f6-4f56-8c45-f7b8ac5084e5 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:04.185514 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 642/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Tue Aug 30 14:42:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:06.068536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da34dfc-16d9-46bd-80f9-e41deb222f46 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:06.069063 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da34dfc-16d9-46bd-80f9-e41deb222f46 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:06.069187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da34dfc-16d9-46bd-80f9-e41deb222f46 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:06.070462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 651/1293] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:06.075584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc3f40-6595-40c2-b272-81da91e504b7 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:06.076067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc3f40-6595-40c2-b272-81da91e504b7 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:06.076158 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc3f40-6595-40c2-b272-81da91e504b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:06.095394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-d7dc3f40-6595-40c2-b272-81da91e504b7 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:06.096599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d7dc3f40-6595-40c2-b272-81da91e504b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:06.125408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:06.162318 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 643/1294] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:42:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:42:06.438701 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da34dfc-16d9-46bd-80f9-e41deb222f46 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:06.470084 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:06.538568 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da34dfc-16d9-46bd-80f9-e41deb222f46 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:06.573351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3413218-0ac3-499c-ae68-03121c667aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:06.573955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3413218-0ac3-499c-ae68-03121c667aea None admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:06.574420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3413218-0ac3-499c-ae68-03121c667aea None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:06.574531 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3413218-0ac3-499c-ae68-03121c667aea None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:06.575197 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3413218-0ac3-499c-ae68-03121c667aea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:06.576574 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3413218-0ac3-499c-ae68-03121c667aea None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:06.660352 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 652/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Aug 30 14:42:06 2022] POST /identity/v3/services => generated 246 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:08.569434 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc3abe-5e43-4487-98da-c2fe66ac5f7c None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:08.569959 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc3abe-5e43-4487-98da-c2fe66ac5f7c None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:08.569959 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc3abe-5e43-4487-98da-c2fe66ac5f7c None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:08.571260 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 644/1296] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:08.576405 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13006d0-95da-4a61-868c-4ee0bcf1e0b6 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:08.576846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13006d0-95da-4a61-868c-4ee0bcf1e0b6 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:08.576964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13006d0-95da-4a61-868c-4ee0bcf1e0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:08.595896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c13006d0-95da-4a61-868c-4ee0bcf1e0b6 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:08.597092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c13006d0-95da-4a61-868c-4ee0bcf1e0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:08.668040 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 653/1297] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:42:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:42:08.941129 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc3abe-5e43-4487-98da-c2fe66ac5f7c None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:09.039949 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc3abe-5e43-4487-98da-c2fe66ac5f7c None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:09.073639 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc55ead-07c6-4e12-9f80-8ccb7aaf45e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:09.074262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc55ead-07c6-4e12-9f80-8ccb7aaf45e2 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.074541 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc55ead-07c6-4e12-9f80-8ccb7aaf45e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.074844 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc55ead-07c6-4e12-9f80-8ccb7aaf45e2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.075410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc55ead-07c6-4e12-9f80-8ccb7aaf45e2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:09.076778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc55ead-07c6-4e12-9f80-8ccb7aaf45e2 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-0dc55ead-07c6-4e12-9f80-8ccb7aaf45e2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:42:09.085907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:42:09.087064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:42:09.087064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:42:09.087064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:42:09.087064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:42:09.087064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:42:09.087064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:42:09.087064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:42:09.087064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:42:09.087064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:42:09.087064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:42:09.087064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:42:09.087064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:42:09.087064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:42:09.087064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:42:09.087064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:42:09.087064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:42:09.088228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:42:09.088228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:42:09.088228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:42:09.088228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:42:09.088228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:42:09.088228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:42:09.088228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:42:09.088228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:42:09.088228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:42:09.088228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:42:09.088228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:42:09.088228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:42:09.088228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:42:09.088228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:42:09.091253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 645/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 30 14:42:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:42:09.092930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13006d0-95da-4a61-868c-4ee0bcf1e0b6 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:09.111499 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:09.190585 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13006d0-95da-4a61-868c-4ee0bcf1e0b6 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:09.222959 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d2d71-f30b-4fd6-9a48-215efbf4ba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:09.223536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d2d71-f30b-4fd6-9a48-215efbf4ba6e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.223818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d2d71-f30b-4fd6-9a48-215efbf4ba6e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.224090 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d2d71-f30b-4fd6-9a48-215efbf4ba6e None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.224715 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d2d71-f30b-4fd6-9a48-215efbf4ba6e None admin] RBAC: Adding query filter params (name=placement) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:09.225017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d2d71-f30b-4fd6-9a48-215efbf4ba6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:09.226383 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d2d71-f30b-4fd6-9a48-215efbf4ba6e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:09.231366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 654/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Tue Aug 30 14:42:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:09.237867 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc55ead-07c6-4e12-9f80-8ccb7aaf45e2 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:09.270751 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:09.339293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc55ead-07c6-4e12-9f80-8ccb7aaf45e2 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:09.373700 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0affbb9-9009-4bd6-aeca-7cd0db3a65eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:09.374290 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0affbb9-9009-4bd6-aeca-7cd0db3a65eb None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.374590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0affbb9-9009-4bd6-aeca-7cd0db3a65eb None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.374857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0affbb9-9009-4bd6-aeca-7cd0db3a65eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.376143 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0affbb9-9009-4bd6-aeca-7cd0db3a65eb None admin] RBAC: Adding query filter params (interface=public, service_id=f7be8f6878b84004b325d7b0d85a3ba3, region_id=RegionOne) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:09.376448 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0affbb9-9009-4bd6-aeca-7cd0db3a65eb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:09.377792 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0affbb9-9009-4bd6-aeca-7cd0db3a65eb None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:09.382822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 646/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Aug 30 14:42:09 2022] GET /identity/v3/endpoints?service_id=f7be8f6878b84004b325d7b0d85a3ba3&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:42:09.388326 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d2d71-f30b-4fd6-9a48-215efbf4ba6e None admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:09.467059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:09.485260 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d2d71-f30b-4fd6-9a48-215efbf4ba6e None admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:09.517521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c8dad-aec5-41ef-b107-402a22fcb48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:09.518104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c8dad-aec5-41ef-b107-402a22fcb48f None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.518377 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c8dad-aec5-41ef-b107-402a22fcb48f None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.518643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c8dad-aec5-41ef-b107-402a22fcb48f None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.519265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c8dad-aec5-41ef-b107-402a22fcb48f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:09.520697 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c8dad-aec5-41ef-b107-402a22fcb48f None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:09.527276 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 655/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Aug 30 14:42:09 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:11.418133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70675cea-b2f8-4178-b9c1-af8f5aec6959 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.418681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70675cea-b2f8-4178-b9c1-af8f5aec6959 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.418681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70675cea-b2f8-4178-b9c1-af8f5aec6959 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.419971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 647/1302] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:11.425644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e427bcd-6d65-4126-a567-10e4ffeaa446 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.426505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e427bcd-6d65-4126-a567-10e4ffeaa446 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.426505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e427bcd-6d65-4126-a567-10e4ffeaa446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.445313 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3e427bcd-6d65-4126-a567-10e4ffeaa446 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:11.446485 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3e427bcd-6d65-4126-a567-10e4ffeaa446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:11.511988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 656/1303] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:42:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:42:11.784586 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70675cea-b2f8-4178-b9c1-af8f5aec6959 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.878261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:11.884755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70675cea-b2f8-4178-b9c1-af8f5aec6959 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.918309 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f22c9-0a84-4e7e-b874-531523b1e1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.918925 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f22c9-0a84-4e7e-b874-531523b1e1b2 None admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.919244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f22c9-0a84-4e7e-b874-531523b1e1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.919515 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f22c9-0a84-4e7e-b874-531523b1e1b2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.920088 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f22c9-0a84-4e7e-b874-531523b1e1b2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.921437 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f22c9-0a84-4e7e-b874-531523b1e1b2 None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application [None req-344f22c9-0a84-4e7e-b874-531523b1e1b2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:42:11.930447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:42:11.931692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:42:11.931692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:42:11.931692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:42:11.931692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:42:11.931692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:42:11.931692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:42:11.931692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:42:11.931692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:42:11.931692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:42:11.931692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:42:11.931692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:42:11.931692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:42:11.931692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:42:11.931692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:42:11.931692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:42:11.931692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:42:11.932816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:42:11.932816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application with Lock( Aug 30 14:42:11.932816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:42:11.932816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:42:11.932816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:42:11.932816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:42:11.932816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:42:11.932816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:42:11.932816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:42:11.932816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:42:11.932816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:42:11.932816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:42:11.932816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:42:11.932816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: ERROR keystone.server.flask.application Aug 30 14:42:11.932816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 648/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Aug 30 14:42:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:42:11.937708 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e427bcd-6d65-4126-a567-10e4ffeaa446 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:12.011698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:12.033998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e427bcd-6d65-4126-a567-10e4ffeaa446 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:12.068048 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba5dfd-0fc7-45e3-ade3-2ea64bcb073e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:12.068582 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba5dfd-0fc7-45e3-ade3-2ea64bcb073e None admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.068860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba5dfd-0fc7-45e3-ade3-2ea64bcb073e None admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.069130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba5dfd-0fc7-45e3-ade3-2ea64bcb073e None admin] PATH_INFO: `/v3/services` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.069816 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba5dfd-0fc7-45e3-ade3-2ea64bcb073e None admin] RBAC: Adding query filter params (name=placement) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:12.070114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba5dfd-0fc7-45e3-ade3-2ea64bcb073e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:12.071516 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba5dfd-0fc7-45e3-ade3-2ea64bcb073e None admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:12.076555 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 657/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Tue Aug 30 14:42:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:12.082826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f22c9-0a84-4e7e-b874-531523b1e1b2 None admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:12.180210 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f22c9-0a84-4e7e-b874-531523b1e1b2 None admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:12.212753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccc1db-4ee5-4f3f-96f1-3e52a70ae93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:12.213334 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccc1db-4ee5-4f3f-96f1-3e52a70ae93a None admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.213610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccc1db-4ee5-4f3f-96f1-3e52a70ae93a None admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.213877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccc1db-4ee5-4f3f-96f1-3e52a70ae93a None admin] PATH_INFO: `/v3/endpoints` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.214527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccc1db-4ee5-4f3f-96f1-3e52a70ae93a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:12.215905 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccc1db-4ee5-4f3f-96f1-3e52a70ae93a None admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:12.321419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 649/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:22.073586 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=30 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:22.316675 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:31.646037 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caa2fb-94c1-4b65-9a47-2d8bb9c8e9bd None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:31.646514 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caa2fb-94c1-4b65-9a47-2d8bb9c8e9bd None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:31.646899 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37caa2fb-94c1-4b65-9a47-2d8bb9c8e9bd None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:31.653718 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176be3b-7069-41c8-88cf-6f98174a9c52 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:31.654109 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176be3b-7069-41c8-88cf-6f98174a9c52 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:31.654230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176be3b-7069-41c8-88cf-6f98174a9c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:31.658996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 658/1307] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:48:31 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:48:31.740789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e176be3b-7069-41c8-88cf-6f98174a9c52 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:48:31.742047 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e176be3b-7069-41c8-88cf-6f98174a9c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:31.883767 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 650/1308] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:48:32.179728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680313f0-5ca1-4251-90d8-633eb3d89a49 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:32.180016 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680313f0-5ca1-4251-90d8-633eb3d89a49 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:32.180481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680313f0-5ca1-4251-90d8-633eb3d89a49 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:32.182116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 659/1309] 10.222.0.27 () {54 vars in 965 bytes} [Tue Aug 30 14:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:48:32.189049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22309f-2019-4beb-8654-369081c8bc24 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:32.189427 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22309f-2019-4beb-8654-369081c8bc24 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:32.189774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22309f-2019-4beb-8654-369081c8bc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:32.273215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-ba22309f-2019-4beb-8654-369081c8bc24 None None] MFA Rules not processed for user `7bff0664e0b44b7c85344a9f3e88f126`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:48:32.274515 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ba22309f-2019-4beb-8654-369081c8bc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:32.296488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:32.358519 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 651/1310] 10.222.0.27 () {58 vars in 1062 bytes} [Tue Aug 30 14:48:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:48:32.383471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680313f0-5ca1-4251-90d8-633eb3d89a49 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:48:32.383471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-680313f0-5ca1-4251-90d8-633eb3d89a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:32.504651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680313f0-5ca1-4251-90d8-633eb3d89a49 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:48:32.587833 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4102dc6-d4d6-4637-a25f-28afe1ec3402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:48:32.589487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4102dc6-d4d6-4637-a25f-28afe1ec3402 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:32.589487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4102dc6-d4d6-4637-a25f-28afe1ec3402 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:32.589670 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4102dc6-d4d6-4637-a25f-28afe1ec3402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:32.591200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a4102dc6-d4d6-4637-a25f-28afe1ec3402 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:32.652115 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:32.711254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4102dc6-d4d6-4637-a25f-28afe1ec3402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:48:32.712937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4102dc6-d4d6-4637-a25f-28afe1ec3402 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:48:32.733931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 660/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:48:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:48:34.983442 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e470d-47e4-4620-bc5b-8a2fc32e2817 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:34.983986 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e470d-47e4-4620-bc5b-8a2fc32e2817 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:34.983986 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e470d-47e4-4620-bc5b-8a2fc32e2817 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:34.985541 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 652/1312] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:48:34.990939 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be30dce-446d-4bce-aafd-b02b22b42d02 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:34.991508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be30dce-446d-4bce-aafd-b02b22b42d02 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:34.991508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be30dce-446d-4bce-aafd-b02b22b42d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:35.072310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-9be30dce-446d-4bce-aafd-b02b22b42d02 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:48:35.072624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9be30dce-446d-4bce-aafd-b02b22b42d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:35.175789 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 661/1313] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:48:35.472605 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e005743-c079-4b59-bd32-4d3a2d94f036 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:35.472912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e005743-c079-4b59-bd32-4d3a2d94f036 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:35.473192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e005743-c079-4b59-bd32-4d3a2d94f036 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:35.474534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 653/1314] 10.222.0.27 () {54 vars in 965 bytes} [Tue Aug 30 14:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:48:35.480579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2febbb3-91d9-4cba-a6fd-f9305cd6daf3 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:35.480869 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2febbb3-91d9-4cba-a6fd-f9305cd6daf3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:35.481144 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2febbb3-91d9-4cba-a6fd-f9305cd6daf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:35.492685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:35.555673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c2febbb3-91d9-4cba-a6fd-f9305cd6daf3 None None] MFA Rules not processed for user `7bff0664e0b44b7c85344a9f3e88f126`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:48:35.556861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c2febbb3-91d9-4cba-a6fd-f9305cd6daf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:35.630709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 662/1315] 10.222.0.27 () {58 vars in 1062 bytes} [Tue Aug 30 14:48:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:48:35.639606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e005743-c079-4b59-bd32-4d3a2d94f036 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:48:35.641159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-4e005743-c079-4b59-bd32-4d3a2d94f036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:35.746119 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e005743-c079-4b59-bd32-4d3a2d94f036 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:48:35.816598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:35.819226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe594ac-ef0e-4367-b7aa-862ab4ce6b2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:48:35.819827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe594ac-ef0e-4367-b7aa-862ab4ce6b2f service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:35.820131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe594ac-ef0e-4367-b7aa-862ab4ce6b2f service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:35.820403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe594ac-ef0e-4367-b7aa-862ab4ce6b2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:35.821991 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9fe594ac-ef0e-4367-b7aa-862ab4ce6b2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:35.932274 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe594ac-ef0e-4367-b7aa-862ab4ce6b2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:48:35.933661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe594ac-ef0e-4367-b7aa-862ab4ce6b2f service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:48:35.951517 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 654/1316] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:48:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:48:38.565277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d0a70-58d4-4fe9-9ad8-ddad1ae5440f None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:38.565815 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d0a70-58d4-4fe9-9ad8-ddad1ae5440f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:38.565815 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d0a70-58d4-4fe9-9ad8-ddad1ae5440f None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:38.567194 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 663/1317] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:48:38.572451 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68954b7-0378-4981-9555-16f1b6396d7b None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:38.572872 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68954b7-0378-4981-9555-16f1b6396d7b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:38.573002 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68954b7-0378-4981-9555-16f1b6396d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:38.638398 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-a68954b7-0378-4981-9555-16f1b6396d7b None None] MFA Rules not processed for user `6827d81000d9402b8bf49e5bf4aed9cd`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:48:38.639633 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a68954b7-0378-4981-9555-16f1b6396d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:38.690236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:38.733922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 655/1318] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:48:38 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:48:39.018434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d0a70-58d4-4fe9-9ad8-ddad1ae5440f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:48:39.019992 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-364d0a70-58d4-4fe9-9ad8-ddad1ae5440f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:39.122720 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d0a70-58d4-4fe9-9ad8-ddad1ae5440f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:48:39.125496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:39.195580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d374944-1a72-4c51-826d-c3363b811b09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:48:39.196179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d374944-1a72-4c51-826d-c3363b811b09 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:39.196456 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d374944-1a72-4c51-826d-c3363b811b09 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:39.196724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d374944-1a72-4c51-826d-c3363b811b09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:39.198318 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4d374944-1a72-4c51-826d-c3363b811b09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:39.305685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d374944-1a72-4c51-826d-c3363b811b09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:48:39.307189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d374944-1a72-4c51-826d-c3363b811b09 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:48:39.325817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 664/1319] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:48:39 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:48:41.988268 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8264c-5955-401d-b807-2a57ea1dec61 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:41.988797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8264c-5955-401d-b807-2a57ea1dec61 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:41.988797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8264c-5955-401d-b807-2a57ea1dec61 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:41.990151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 656/1320] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:48:41.995216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28bc81-bd57-4e0c-8662-34edd2c894f7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:41.996114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28bc81-bd57-4e0c-8662-34edd2c894f7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:41.996251 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28bc81-bd57-4e0c-8662-34edd2c894f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:42.036720 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:42.062495 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4c28bc81-bd57-4e0c-8662-34edd2c894f7 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:48:42.063786 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4c28bc81-bd57-4e0c-8662-34edd2c894f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:42.157530 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 665/1321] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:48:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:48:42.444066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8264c-5955-401d-b807-2a57ea1dec61 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:48:42.445611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-bab8264c-5955-401d-b807-2a57ea1dec61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:42.548653 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8264c-5955-401d-b807-2a57ea1dec61 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:48:42.620668 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d64500-4096-40c8-8f84-af75750df625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:48:42.621261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d64500-4096-40c8-8f84-af75750df625 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:42.621533 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d64500-4096-40c8-8f84-af75750df625 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:42.621796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d64500-4096-40c8-8f84-af75750df625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:42.623394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-11d64500-4096-40c8-8f84-af75750df625 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:42.630759 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:42.732392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d64500-4096-40c8-8f84-af75750df625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:48:42.733806 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d64500-4096-40c8-8f84-af75750df625 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:48:42.751169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 657/1322] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:48:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:48:45.559703 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d5555-59f3-4cb0-adf5-e989abad0a22 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:45.560216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d5555-59f3-4cb0-adf5-e989abad0a22 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:45.560216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d5555-59f3-4cb0-adf5-e989abad0a22 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:45.561531 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 666/1323] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:48:45.566603 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f336962-68e9-461e-88ab-5e95ac88c459 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:45.567024 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f336962-68e9-461e-88ab-5e95ac88c459 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:45.567177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f336962-68e9-461e-88ab-5e95ac88c459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:45.633401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-0f336962-68e9-461e-88ab-5e95ac88c459 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:48:45.634674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0f336962-68e9-461e-88ab-5e95ac88c459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:45.692253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:45.731927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 658/1324] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:48:46.014478 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d5555-59f3-4cb0-adf5-e989abad0a22 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:48:46.016549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b04d5555-59f3-4cb0-adf5-e989abad0a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:46.120665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d5555-59f3-4cb0-adf5-e989abad0a22 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:48:46.133275 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:46.196363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2de94-39ba-43c6-a4c2-2b038105f27e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:48:46.196993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2de94-39ba-43c6-a4c2-2b038105f27e service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:46.197270 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2de94-39ba-43c6-a4c2-2b038105f27e service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:46.197540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2de94-39ba-43c6-a4c2-2b038105f27e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:46.199702 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9dd2de94-39ba-43c6-a4c2-2b038105f27e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:46.312984 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2de94-39ba-43c6-a4c2-2b038105f27e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:48:46.314344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2de94-39ba-43c6-a4c2-2b038105f27e service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:48:46.333245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 667/1325] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:48:48.301120 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9640887-538f-4993-a58e-278c2bf625df None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:48.301584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9640887-538f-4993-a58e-278c2bf625df None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:48.301673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9640887-538f-4993-a58e-278c2bf625df None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:48.303011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 659/1326] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:48:48.308260 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bea687-73e8-46fe-a7f3-005badaa16ae None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:48.308723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bea687-73e8-46fe-a7f3-005badaa16ae None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:48.308902 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bea687-73e8-46fe-a7f3-005badaa16ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:48.356596 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:48.374070 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-85bea687-73e8-46fe-a7f3-005badaa16ae None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:48:48.375295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-85bea687-73e8-46fe-a7f3-005badaa16ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:48.470574 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 668/1327] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:48:48.750308 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9640887-538f-4993-a58e-278c2bf625df None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:48:48.751924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c9640887-538f-4993-a58e-278c2bf625df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:48.858498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9640887-538f-4993-a58e-278c2bf625df None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:48:48.934618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89169355-7ce8-49f8-a044-2a613873f8e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:48:48.935274 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89169355-7ce8-49f8-a044-2a613873f8e2 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:48.935550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89169355-7ce8-49f8-a044-2a613873f8e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:48.935820 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89169355-7ce8-49f8-a044-2a613873f8e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:48.937436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-89169355-7ce8-49f8-a044-2a613873f8e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:48.958903 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:49.054381 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89169355-7ce8-49f8-a044-2a613873f8e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:48:49.055854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89169355-7ce8-49f8-a044-2a613873f8e2 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:48:49.074937 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 660/1328] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:48:51.046560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f7362-7b33-4f7c-b55f-4f09bea6ee7a None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:51.047054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f7362-7b33-4f7c-b55f-4f09bea6ee7a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:51.047054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f7362-7b33-4f7c-b55f-4f09bea6ee7a None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:51.048436 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 669/1329] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:48:51.054006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972800a-e629-4921-85b6-3052a442d2ce None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:51.054495 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972800a-e629-4921-85b6-3052a442d2ce None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:51.054586 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972800a-e629-4921-85b6-3052a442d2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:51.132000 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-2972800a-e629-4921-85b6-3052a442d2ce None None] MFA Rules not processed for user `6827d81000d9402b8bf49e5bf4aed9cd`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:48:51.133254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-2972800a-e629-4921-85b6-3052a442d2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:51.200857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:51.234569 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 661/1330] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:48:51 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:48:51.512933 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f7362-7b33-4f7c-b55f-4f09bea6ee7a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:48:51.519799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c14f7362-7b33-4f7c-b55f-4f09bea6ee7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:51.637831 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f7362-7b33-4f7c-b55f-4f09bea6ee7a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:48:51.658215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:51.712851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1d22a-4a3c-4183-9ddd-3e9f19f1d501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:48:51.713452 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1d22a-4a3c-4183-9ddd-3e9f19f1d501 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:51.714287 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1d22a-4a3c-4183-9ddd-3e9f19f1d501 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:51.714557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1d22a-4a3c-4183-9ddd-3e9f19f1d501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:51.716256 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-12f1d22a-4a3c-4183-9ddd-3e9f19f1d501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:51.822276 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1d22a-4a3c-4183-9ddd-3e9f19f1d501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:48:51.824210 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1d22a-4a3c-4183-9ddd-3e9f19f1d501 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:48:51.842319 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 670/1331] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:48:51 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:48:54.216295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ad51f-c3e2-4052-83e2-fda5fb63a1f6 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:54.216852 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ad51f-c3e2-4052-83e2-fda5fb63a1f6 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:54.216852 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ad51f-c3e2-4052-83e2-fda5fb63a1f6 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:54.218167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 662/1332] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:48:54.223194 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687dac4e-0567-4026-a54f-c2feb4bea7b6 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:54.223615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687dac4e-0567-4026-a54f-c2feb4bea7b6 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:54.223745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687dac4e-0567-4026-a54f-c2feb4bea7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:54.287958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:54.290549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-687dac4e-0567-4026-a54f-c2feb4bea7b6 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:48:54.292303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-687dac4e-0567-4026-a54f-c2feb4bea7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:54.387777 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 671/1333] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:48:54.669302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ad51f-c3e2-4052-83e2-fda5fb63a1f6 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:48:54.670890 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ef0ad51f-c3e2-4052-83e2-fda5fb63a1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:54.772210 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ad51f-c3e2-4052-83e2-fda5fb63a1f6 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:48:54.843755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42176ea1-eef6-40a7-8928-fe01185cfd02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:48:54.844349 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42176ea1-eef6-40a7-8928-fe01185cfd02 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:54.844626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42176ea1-eef6-40a7-8928-fe01185cfd02 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:54.844895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42176ea1-eef6-40a7-8928-fe01185cfd02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:54.846470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-42176ea1-eef6-40a7-8928-fe01185cfd02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:54.872579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:54.952980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42176ea1-eef6-40a7-8928-fe01185cfd02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:48:54.954331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42176ea1-eef6-40a7-8928-fe01185cfd02 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:48:54.972854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 663/1334] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:48:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:48:56.916963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bcdfc-759a-420f-83a3-528db6b8bc73 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:56.917497 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bcdfc-759a-420f-83a3-528db6b8bc73 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:56.917497 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bcdfc-759a-420f-83a3-528db6b8bc73 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:56.918842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 672/1335] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:48:56.924022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed6fc3-74e7-4e90-a6e3-6efa4fcf8dbd None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:56.924462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed6fc3-74e7-4e90-a6e3-6efa4fcf8dbd None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:56.924556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed6fc3-74e7-4e90-a6e3-6efa4fcf8dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:56.989303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-50ed6fc3-74e7-4e90-a6e3-6efa4fcf8dbd None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:48:56.990515 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-50ed6fc3-74e7-4e90-a6e3-6efa4fcf8dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:57.065660 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:57.087269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 664/1336] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:48:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:48:57.371189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bcdfc-759a-420f-83a3-528db6b8bc73 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:48:57.372717 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-199bcdfc-759a-420f-83a3-528db6b8bc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:57.474347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bcdfc-759a-420f-83a3-528db6b8bc73 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:48:57.501762 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:57.546446 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45428f-30c3-4377-911e-7916cfd705ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:48:57.546969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45428f-30c3-4377-911e-7916cfd705ab service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:57.547269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45428f-30c3-4377-911e-7916cfd705ab service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:57.547536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45428f-30c3-4377-911e-7916cfd705ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:57.549094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4c45428f-30c3-4377-911e-7916cfd705ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:57.653681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c45428f-30c3-4377-911e-7916cfd705ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:48:57.655381 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c45428f-30c3-4377-911e-7916cfd705ab service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:48:57.673689 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 673/1337] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:48:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:00.318497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32e949-bfd6-43c5-bcf2-bdbdd7008c48 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:00.319018 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32e949-bfd6-43c5-bcf2-bdbdd7008c48 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:00.319018 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32e949-bfd6-43c5-bcf2-bdbdd7008c48 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:00.320404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 665/1338] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:00.325940 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06314f51-3db4-4fb9-b086-c534b2f8e301 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:00.326385 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06314f51-3db4-4fb9-b086-c534b2f8e301 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:00.326493 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06314f51-3db4-4fb9-b086-c534b2f8e301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:00.393171 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-06314f51-3db4-4fb9-b086-c534b2f8e301 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:00.394368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-06314f51-3db4-4fb9-b086-c534b2f8e301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:00.402624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:00.489116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 674/1339] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:00.772666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32e949-bfd6-43c5-bcf2-bdbdd7008c48 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:00.774224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9b32e949-bfd6-43c5-bcf2-bdbdd7008c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:00.877827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32e949-bfd6-43c5-bcf2-bdbdd7008c48 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:00.953845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e65b1-bf60-4d9e-9949-940993d07073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:00.954453 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e65b1-bf60-4d9e-9949-940993d07073 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:00.954740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e65b1-bf60-4d9e-9949-940993d07073 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:00.955175 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e65b1-bf60-4d9e-9949-940993d07073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:00.956813 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-3d6e65b1-bf60-4d9e-9949-940993d07073 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:00.992428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:01.065446 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e65b1-bf60-4d9e-9949-940993d07073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:01.066809 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e65b1-bf60-4d9e-9949-940993d07073 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:01.085355 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 666/1340] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:05.127985 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba27719-b156-447a-b5dd-04ac5cf25617 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:05.128545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba27719-b156-447a-b5dd-04ac5cf25617 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:05.128545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba27719-b156-447a-b5dd-04ac5cf25617 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:05.130037 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 675/1341] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:05.135146 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad41104-f2cc-4175-a84b-2b96de6c522a None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:05.135587 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad41104-f2cc-4175-a84b-2b96de6c522a None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:05.135773 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad41104-f2cc-4175-a84b-2b96de6c522a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:05.201593 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-6ad41104-f2cc-4175-a84b-2b96de6c522a None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:05.202806 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6ad41104-f2cc-4175-a84b-2b96de6c522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:05.296061 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:05.307515 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 667/1342] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:05.596677 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba27719-b156-447a-b5dd-04ac5cf25617 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:05.598230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fba27719-b156-447a-b5dd-04ac5cf25617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:05.701666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba27719-b156-447a-b5dd-04ac5cf25617 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:05.739434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:05.774436 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8abe3-1b3b-4985-975b-0f84f9ad4eba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:05.775033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8abe3-1b3b-4985-975b-0f84f9ad4eba service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:05.775335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8abe3-1b3b-4985-975b-0f84f9ad4eba service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:05.775608 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8abe3-1b3b-4985-975b-0f84f9ad4eba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:05.777700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9ab8abe3-1b3b-4985-975b-0f84f9ad4eba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:05.888641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8abe3-1b3b-4985-975b-0f84f9ad4eba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:05.890005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8abe3-1b3b-4985-975b-0f84f9ad4eba service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:05.909346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 676/1343] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:09.060304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d631de-ecfa-4b75-9f0e-b6eba0ec00cb None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:09.060882 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d631de-ecfa-4b75-9f0e-b6eba0ec00cb None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:09.060882 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d631de-ecfa-4b75-9f0e-b6eba0ec00cb None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:09.067564 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfc9d6-e810-46cb-8d82-68a7e5c4fca3 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:09.067976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfc9d6-e810-46cb-8d82-68a7e5c4fca3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:09.068087 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfc9d6-e810-46cb-8d82-68a7e5c4fca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:09.072824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 668/1344] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:09 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:09.137314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-dccfc9d6-e810-46cb-8d82-68a7e5c4fca3 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:09.138519 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-dccfc9d6-e810-46cb-8d82-68a7e5c4fca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:09.158029 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:09.250293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 677/1345] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:49:09.537236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d631de-ecfa-4b75-9f0e-b6eba0ec00cb None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:09.538764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-f5d631de-ecfa-4b75-9f0e-b6eba0ec00cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:09.646305 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d631de-ecfa-4b75-9f0e-b6eba0ec00cb None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:09.719977 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db120e-5bc6-4dd9-bfc9-1c8b6f592847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:09.720603 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db120e-5bc6-4dd9-bfc9-1c8b6f592847 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:09.720862 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db120e-5bc6-4dd9-bfc9-1c8b6f592847 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:09.721131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db120e-5bc6-4dd9-bfc9-1c8b6f592847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:09.722727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-94db120e-5bc6-4dd9-bfc9-1c8b6f592847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:09.766590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:09.834094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db120e-5bc6-4dd9-bfc9-1c8b6f592847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:09.835558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db120e-5bc6-4dd9-bfc9-1c8b6f592847 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:09.854154 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 669/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:14.152073 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00d490-4f36-441e-a7da-1c2811b5d1e3 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:14.152549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00d490-4f36-441e-a7da-1c2811b5d1e3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:14.152639 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00d490-4f36-441e-a7da-1c2811b5d1e3 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:14.153964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 678/1347] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:14.159719 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56810c1c-50f7-444b-b112-5115558cacad None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:14.160237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56810c1c-50f7-444b-b112-5115558cacad None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:14.160237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56810c1c-50f7-444b-b112-5115558cacad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:14.230754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-56810c1c-50f7-444b-b112-5115558cacad None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:14.234108 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-56810c1c-50f7-444b-b112-5115558cacad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:14.334796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:14.347285 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 670/1348] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:14.623890 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00d490-4f36-441e-a7da-1c2811b5d1e3 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:14.625495 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9d00d490-4f36-441e-a7da-1c2811b5d1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:14.728503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00d490-4f36-441e-a7da-1c2811b5d1e3 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:14.774545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:14.800820 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f66e57-e9f6-401d-b192-aaf888ec0d90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:14.801414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f66e57-e9f6-401d-b192-aaf888ec0d90 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:14.801688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f66e57-e9f6-401d-b192-aaf888ec0d90 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:14.801951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f66e57-e9f6-401d-b192-aaf888ec0d90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:14.803557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f2f66e57-e9f6-401d-b192-aaf888ec0d90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:14.909737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f66e57-e9f6-401d-b192-aaf888ec0d90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:14.911180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f66e57-e9f6-401d-b192-aaf888ec0d90 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:14.929675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 679/1349] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:19.839241 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49f991-d671-422a-893f-7241dacc6000 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:19.839677 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49f991-d671-422a-893f-7241dacc6000 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:19.839770 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49f991-d671-422a-893f-7241dacc6000 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:19.841398 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 671/1350] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 30 14:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:19.847569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50921d36-5881-4dd2-959d-761b411cbe53 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:19.847998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50921d36-5881-4dd2-959d-761b411cbe53 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:19.848117 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50921d36-5881-4dd2-959d-761b411cbe53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:19.965158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-50921d36-5881-4dd2-959d-761b411cbe53 None None] MFA Rules not processed for user `f4f11655cd51438e9652869f74ffa975`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:19.971022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-50921d36-5881-4dd2-959d-761b411cbe53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:20.000447 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:20.106029 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 680/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 30 14:49:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:49:24.192367 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258f1d3-a9fa-4507-8202-d7d7818635a0 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:24.192925 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258f1d3-a9fa-4507-8202-d7d7818635a0 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:24.193196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258f1d3-a9fa-4507-8202-d7d7818635a0 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:24.194807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 672/1352] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 30 14:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:24.199795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f7104-9767-41cb-9e7d-9d8cc69215d7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:24.200518 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f7104-9767-41cb-9e7d-9d8cc69215d7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:24.200978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f7104-9767-41cb-9e7d-9d8cc69215d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:24.293306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a54f7104-9767-41cb-9e7d-9d8cc69215d7 None None] MFA Rules not processed for user `f4f11655cd51438e9652869f74ffa975`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:24.294796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a54f7104-9767-41cb-9e7d-9d8cc69215d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:24.391751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:24.412138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 681/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 30 14:49:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:25.838211 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5d1a6-e685-4839-ad2c-83ea4b0366bf None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:25.839394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5d1a6-e685-4839-ad2c-83ea4b0366bf None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:25.839394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5d1a6-e685-4839-ad2c-83ea4b0366bf None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:25.841073 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 673/1354] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 30 14:49:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:25.846465 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878831b5-34fe-4df1-89dd-fe10c668c72b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:25.846906 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878831b5-34fe-4df1-89dd-fe10c668c72b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:25.847025 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878831b5-34fe-4df1-89dd-fe10c668c72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:25.923663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-878831b5-34fe-4df1-89dd-fe10c668c72b None None] MFA Rules not processed for user `f4f11655cd51438e9652869f74ffa975`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:25.924876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-878831b5-34fe-4df1-89dd-fe10c668c72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:26.031605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 682/1355] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 30 14:49:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:29.283031 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31deff75-fb27-4e54-a12a-69b2bd6d5419 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:29.284432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31deff75-fb27-4e54-a12a-69b2bd6d5419 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:29.284432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31deff75-fb27-4e54-a12a-69b2bd6d5419 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:29.284944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 674/1356] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:29.289973 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753f8b7-97b7-45ca-bd08-72d27a3cf234 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:29.290386 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753f8b7-97b7-45ca-bd08-72d27a3cf234 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:29.290522 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753f8b7-97b7-45ca-bd08-72d27a3cf234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:29.334771 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:29.370191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b753f8b7-97b7-45ca-bd08-72d27a3cf234 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:29.374835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b753f8b7-97b7-45ca-bd08-72d27a3cf234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:29.512864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 683/1357] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:29.811536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f14a6-2b5d-4f04-8a30-4af48839a366 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:29.812490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f14a6-2b5d-4f04-8a30-4af48839a366 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:29.813370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f14a6-2b5d-4f04-8a30-4af48839a366 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:29.816831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 675/1358] 10.222.0.27 () {54 vars in 962 bytes} [Tue Aug 30 14:49:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:29.824603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a06a02-da88-4305-995e-5a5949ee338c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:29.824984 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a06a02-da88-4305-995e-5a5949ee338c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:29.825255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a06a02-da88-4305-995e-5a5949ee338c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:29.910755 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-80a06a02-da88-4305-995e-5a5949ee338c None None] MFA Rules not processed for user `5b6e4c5bf2a54b28a55e5ae74b90ab4f`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:29.912658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-80a06a02-da88-4305-995e-5a5949ee338c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:29.965780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:30.037103 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 684/1359] 10.222.0.27 () {58 vars in 1059 bytes} [Tue Aug 30 14:49:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:30.045493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f14a6-2b5d-4f04-8a30-4af48839a366 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:30.047128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ba7f14a6-2b5d-4f04-8a30-4af48839a366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:30.218859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f14a6-2b5d-4f04-8a30-4af48839a366 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:30.358601 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:30.358601 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e088b42-aa78-4437-bec4-73f2febe2518 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:30.358601 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e088b42-aa78-4437-bec4-73f2febe2518 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:30.358601 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e088b42-aa78-4437-bec4-73f2febe2518 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:30.359699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e088b42-aa78-4437-bec4-73f2febe2518 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:30.359699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9e088b42-aa78-4437-bec4-73f2febe2518 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:30.446084 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e088b42-aa78-4437-bec4-73f2febe2518 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:30.447787 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e088b42-aa78-4437-bec4-73f2febe2518 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:30.466590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 676/1360] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:32.614477 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c943c3-3cb4-4889-82d2-9ba3f3c4a6b1 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:32.614988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c943c3-3cb4-4889-82d2-9ba3f3c4a6b1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:32.614988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c943c3-3cb4-4889-82d2-9ba3f3c4a6b1 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:32.616635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 685/1361] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:32.622318 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea19ca-1647-4a11-b172-0082cce5730a None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:32.622759 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea19ca-1647-4a11-b172-0082cce5730a None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:32.622931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea19ca-1647-4a11-b172-0082cce5730a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:32.695517 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-eaea19ca-1647-4a11-b172-0082cce5730a None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:32.696712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-eaea19ca-1647-4a11-b172-0082cce5730a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:32.736142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:32.801870 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 677/1362] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:33.099306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c943c3-3cb4-4889-82d2-9ba3f3c4a6b1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:33.101286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d8c943c3-3cb4-4889-82d2-9ba3f3c4a6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:33.261491 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c943c3-3cb4-4889-82d2-9ba3f3c4a6b1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:33.272097 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:33.384244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503fa21-3eae-41fc-8010-fbc1563b1e35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:33.384929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503fa21-3eae-41fc-8010-fbc1563b1e35 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:33.385328 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503fa21-3eae-41fc-8010-fbc1563b1e35 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:33.385660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503fa21-3eae-41fc-8010-fbc1563b1e35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:33.388226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8503fa21-3eae-41fc-8010-fbc1563b1e35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:33.492385 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503fa21-3eae-41fc-8010-fbc1563b1e35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:33.494161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503fa21-3eae-41fc-8010-fbc1563b1e35 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:33.515189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 686/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:49:35.899724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc8d5e-3874-45b1-b20f-31948d0942a7 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:35.900322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc8d5e-3874-45b1-b20f-31948d0942a7 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:35.900322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc8d5e-3874-45b1-b20f-31948d0942a7 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:35.902547 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 678/1364] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:35.907577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49785b-9af9-406d-8630-b0df49822999 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:35.908022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49785b-9af9-406d-8630-b0df49822999 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:35.908106 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49785b-9af9-406d-8630-b0df49822999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:35.925433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:35.983145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ca49785b-9af9-406d-8630-b0df49822999 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:35.984359 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ca49785b-9af9-406d-8630-b0df49822999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:36.088637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 687/1365] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:36.375944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a644ea-67ec-40ec-8f88-e082f85e445d None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:36.376967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a644ea-67ec-40ec-8f88-e082f85e445d None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:36.377349 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a644ea-67ec-40ec-8f88-e082f85e445d None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:36.379139 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 679/1366] 10.222.0.27 () {54 vars in 962 bytes} [Tue Aug 30 14:49:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:36.384524 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521020c2-fed9-427f-9217-af423f651041 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:36.384755 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521020c2-fed9-427f-9217-af423f651041 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:36.385050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521020c2-fed9-427f-9217-af423f651041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:36.466818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-521020c2-fed9-427f-9217-af423f651041 None None] MFA Rules not processed for user `5b6e4c5bf2a54b28a55e5ae74b90ab4f`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:36.468230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-521020c2-fed9-427f-9217-af423f651041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:36.508259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:36.596553 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 688/1367] 10.222.0.27 () {58 vars in 1059 bytes} [Tue Aug 30 14:49:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:36.605060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a644ea-67ec-40ec-8f88-e082f85e445d None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:36.606574 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b3a644ea-67ec-40ec-8f88-e082f85e445d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:36.764152 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a644ea-67ec-40ec-8f88-e082f85e445d None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:36.786500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:36.890531 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b52339-56e7-41bb-978b-35dc499f87fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:36.892488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b52339-56e7-41bb-978b-35dc499f87fe service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:36.892488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b52339-56e7-41bb-978b-35dc499f87fe service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:36.892488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b52339-56e7-41bb-978b-35dc499f87fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:36.893493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b3b52339-56e7-41bb-978b-35dc499f87fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:36.989053 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b52339-56e7-41bb-978b-35dc499f87fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:36.990665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b52339-56e7-41bb-978b-35dc499f87fe service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:37.010316 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 680/1368] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:37.415379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d187b-c9d9-444d-b5be-fcd2d531f1d5 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:37.415937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d187b-c9d9-444d-b5be-fcd2d531f1d5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:37.415937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d187b-c9d9-444d-b5be-fcd2d531f1d5 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:37.418005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 689/1369] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:37.422998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24ee64-6526-4b15-aae2-cae14f3abdb3 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:37.423326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24ee64-6526-4b15-aae2-cae14f3abdb3 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:37.423602 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24ee64-6526-4b15-aae2-cae14f3abdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:37.457086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:37.492090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-9f24ee64-6526-4b15-aae2-cae14f3abdb3 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:37.493701 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9f24ee64-6526-4b15-aae2-cae14f3abdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:37.595126 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 681/1370] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:38.136404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ee9fe-bf70-4e8a-be9e-5a4e605c255d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:38.136873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ee9fe-bf70-4e8a-be9e-5a4e605c255d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:38.137003 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ee9fe-bf70-4e8a-be9e-5a4e605c255d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:38.138463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 690/1371] 10.222.0.27 () {54 vars in 964 bytes} [Tue Aug 30 14:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:38.143197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bd903-631d-4a49-af04-fe03e9565878 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:38.143630 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bd903-631d-4a49-af04-fe03e9565878 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:38.143742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bd903-631d-4a49-af04-fe03e9565878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:38.224079 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-ed0bd903-631d-4a49-af04-fe03e9565878 None None] MFA Rules not processed for user `ef389656cc8546fdb7d9f30329b93dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:38.225279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ed0bd903-631d-4a49-af04-fe03e9565878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:38.393827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:38.424486 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 682/1372] 10.222.0.27 () {58 vars in 1061 bytes} [Tue Aug 30 14:49:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:38.431883 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ee9fe-bf70-4e8a-be9e-5a4e605c255d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:38.433389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-143ee9fe-bf70-4e8a-be9e-5a4e605c255d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:38.542613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ee9fe-bf70-4e8a-be9e-5a4e605c255d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:38.587192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:38.622411 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe09d08-e817-440f-8f63-79bee38820b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:38.623016 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe09d08-e817-440f-8f63-79bee38820b5 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:38.623327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe09d08-e817-440f-8f63-79bee38820b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:38.623601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe09d08-e817-440f-8f63-79bee38820b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:38.625191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-efe09d08-e817-440f-8f63-79bee38820b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:38.763503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe09d08-e817-440f-8f63-79bee38820b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:38.765554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe09d08-e817-440f-8f63-79bee38820b5 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:38.797451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 691/1373] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:39.143435 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a34c6-ee29-4e58-ac95-85618da0a0c2 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:39.143984 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a34c6-ee29-4e58-ac95-85618da0a0c2 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:39.143984 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a34c6-ee29-4e58-ac95-85618da0a0c2 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:39.145469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 683/1374] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:39.153361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccd5ec-d819-4b4b-9d2f-16a3e100c590 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:39.153361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccd5ec-d819-4b4b-9d2f-16a3e100c590 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:39.153361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccd5ec-d819-4b4b-9d2f-16a3e100c590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:39.217892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-04ccd5ec-d819-4b4b-9d2f-16a3e100c590 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:39.219379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-04ccd5ec-d819-4b4b-9d2f-16a3e100c590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:39.241098 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:39.319835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 692/1375] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:39.617123 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a34c6-ee29-4e58-ac95-85618da0a0c2 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:39.618632 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-5f9a34c6-ee29-4e58-ac95-85618da0a0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:39.764573 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a34c6-ee29-4e58-ac95-85618da0a0c2 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:39.814843 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:39.875632 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a327b3-ed17-4c24-af5a-c3c4cf2346e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:39.876229 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a327b3-ed17-4c24-af5a-c3c4cf2346e8 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:39.876503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a327b3-ed17-4c24-af5a-c3c4cf2346e8 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:39.876777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a327b3-ed17-4c24-af5a-c3c4cf2346e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:39.878631 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-17a327b3-ed17-4c24-af5a-c3c4cf2346e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:39.966135 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a327b3-ed17-4c24-af5a-c3c4cf2346e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:39.967822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a327b3-ed17-4c24-af5a-c3c4cf2346e8 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:39.985941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 684/1376] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:42.047481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4700e-59cb-469f-b6c1-89c1c12256bb None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:42.048344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4700e-59cb-469f-b6c1-89c1c12256bb None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:42.049355 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4700e-59cb-469f-b6c1-89c1c12256bb None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:42.052885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 693/1377] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:42.059034 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cac062-7598-4132-bff6-ee3b592eeb5a None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:42.059879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cac062-7598-4132-bff6-ee3b592eeb5a None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:42.060310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cac062-7598-4132-bff6-ee3b592eeb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:42.137001 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-d6cac062-7598-4132-bff6-ee3b592eeb5a None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:42.138376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d6cac062-7598-4132-bff6-ee3b592eeb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:42.145228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:42.237243 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 685/1378] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:42.526312 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4700e-59cb-469f-b6c1-89c1c12256bb None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:42.528135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c9a4700e-59cb-469f-b6c1-89c1c12256bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:42.690690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4700e-59cb-469f-b6c1-89c1c12256bb None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:42.694988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:42.816261 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02c16e-fda7-4269-b3a0-9b1b6af29cce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:42.816868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02c16e-fda7-4269-b3a0-9b1b6af29cce service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:42.817146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02c16e-fda7-4269-b3a0-9b1b6af29cce service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:42.817420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02c16e-fda7-4269-b3a0-9b1b6af29cce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:42.819141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-cf02c16e-fda7-4269-b3a0-9b1b6af29cce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:42.920178 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02c16e-fda7-4269-b3a0-9b1b6af29cce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:42.921792 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02c16e-fda7-4269-b3a0-9b1b6af29cce service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:42.943560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 694/1379] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:45.025726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636f9c0-0f12-4d15-bd44-210288936eb7 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:45.026250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636f9c0-0f12-4d15-bd44-210288936eb7 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:45.026250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636f9c0-0f12-4d15-bd44-210288936eb7 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:45.027842 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 686/1380] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:49:45.032795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b699e-9815-4031-8191-3eb2a310f0b9 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:45.033201 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b699e-9815-4031-8191-3eb2a310f0b9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:45.033375 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b699e-9815-4031-8191-3eb2a310f0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:45.045467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:45.101415 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-db1b699e-9815-4031-8191-3eb2a310f0b9 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:45.102619 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-db1b699e-9815-4031-8191-3eb2a310f0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:45.206310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 695/1381] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:45.500151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636f9c0-0f12-4d15-bd44-210288936eb7 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:45.501744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-2636f9c0-0f12-4d15-bd44-210288936eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:45.646262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:45.656506 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636f9c0-0f12-4d15-bd44-210288936eb7 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:45.776666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c7414-5dbd-4381-9b10-c303dd7bd6ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:45.777260 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c7414-5dbd-4381-9b10-c303dd7bd6ac service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:45.777537 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c7414-5dbd-4381-9b10-c303dd7bd6ac service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:45.777807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c7414-5dbd-4381-9b10-c303dd7bd6ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:45.779420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a16c7414-5dbd-4381-9b10-c303dd7bd6ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:45.878376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c7414-5dbd-4381-9b10-c303dd7bd6ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:45.879987 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c7414-5dbd-4381-9b10-c303dd7bd6ac service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:45.890681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:45.902156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 687/1382] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:47.999923 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbf578-3f32-4d3b-aa8e-4b6ee4583929 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:48.000452 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbf578-3f32-4d3b-aa8e-4b6ee4583929 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:48.000903 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbf578-3f32-4d3b-aa8e-4b6ee4583929 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:48.003566 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 696/1383] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:48.008953 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76694e-ae36-4ed1-a4cc-f8cb9a1928eb None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:48.009871 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76694e-ae36-4ed1-a4cc-f8cb9a1928eb None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:48.010009 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76694e-ae36-4ed1-a4cc-f8cb9a1928eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:48.078121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-4e76694e-ae36-4ed1-a4cc-f8cb9a1928eb None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:48.079401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-4e76694e-ae36-4ed1-a4cc-f8cb9a1928eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:48.191429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 688/1384] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:48.558520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dbf578-3f32-4d3b-aa8e-4b6ee4583929 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:48.560072 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-61dbf578-3f32-4d3b-aa8e-4b6ee4583929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:48.676729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:48.745759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dbf578-3f32-4d3b-aa8e-4b6ee4583929 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:48.863169 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1383ce-64c5-45d3-90ef-0306d89e7c3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:48.863699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1383ce-64c5-45d3-90ef-0306d89e7c3a service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:48.864006 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1383ce-64c5-45d3-90ef-0306d89e7c3a service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:48.864247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1383ce-64c5-45d3-90ef-0306d89e7c3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:48.865842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4f1383ce-64c5-45d3-90ef-0306d89e7c3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:48.927218 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:48.978288 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1383ce-64c5-45d3-90ef-0306d89e7c3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:48.980602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1383ce-64c5-45d3-90ef-0306d89e7c3a service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:49.003731 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 697/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:50.894995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8ed99-2516-4086-ac82-ea1d5d293bf8 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:50.895967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8ed99-2516-4086-ac82-ea1d5d293bf8 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:50.895967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8ed99-2516-4086-ac82-ea1d5d293bf8 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:50.896812 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 689/1386] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:50.902783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e3d5f-b8b8-4bcd-b2ee-9baf9e6e7a7e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:50.903250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e3d5f-b8b8-4bcd-b2ee-9baf9e6e7a7e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:50.903355 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e3d5f-b8b8-4bcd-b2ee-9baf9e6e7a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:50.979149 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-d63e3d5f-b8b8-4bcd-b2ee-9baf9e6e7a7e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:50.980528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d63e3d5f-b8b8-4bcd-b2ee-9baf9e6e7a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:51.085963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 698/1387] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:51.387400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8ed99-2516-4086-ac82-ea1d5d293bf8 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:51.389014 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8af8ed99-2516-4086-ac82-ea1d5d293bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:51.468888 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:51.542677 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8ed99-2516-4086-ac82-ea1d5d293bf8 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:51.684081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f0fa3-f393-48e6-b6ee-408fe2ab3e77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:51.684726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f0fa3-f393-48e6-b6ee-408fe2ab3e77 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:51.685030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f0fa3-f393-48e6-b6ee-408fe2ab3e77 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:51.685298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f0fa3-f393-48e6-b6ee-408fe2ab3e77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:51.686916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-109f0fa3-f393-48e6-b6ee-408fe2ab3e77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:51.727757 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:51.788232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f0fa3-f393-48e6-b6ee-408fe2ab3e77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:51.789808 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f0fa3-f393-48e6-b6ee-408fe2ab3e77 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:51.809125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 690/1388] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:49:53.852593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca6e0c-7148-48bb-8f5a-ea0d1e7cd01c None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:53.853739 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca6e0c-7148-48bb-8f5a-ea0d1e7cd01c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:53.853739 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca6e0c-7148-48bb-8f5a-ea0d1e7cd01c None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:53.855232 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 699/1389] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:53.860089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ca9aa-eb68-4d05-95ed-778513a9581e None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:53.860491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ca9aa-eb68-4d05-95ed-778513a9581e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:53.860637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ca9aa-eb68-4d05-95ed-778513a9581e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:53.926484 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-fd6ca9aa-eb68-4d05-95ed-778513a9581e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:53.927726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-fd6ca9aa-eb68-4d05-95ed-778513a9581e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:54.025820 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 691/1390] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:54.314386 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca6e0c-7148-48bb-8f5a-ea0d1e7cd01c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:54.315975 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a3ca6e0c-7148-48bb-8f5a-ea0d1e7cd01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:54.326750 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae926773-41f4-4289-9fa3-5c851476e3cb None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:54.327257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae926773-41f4-4289-9fa3-5c851476e3cb None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:54.327534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae926773-41f4-4289-9fa3-5c851476e3cb None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:54.329000 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 692/1391] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:49:54.333784 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f31f7-cf85-4227-ac68-1e3cb4c3777f None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:54.334182 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f31f7-cf85-4227-ac68-1e3cb4c3777f None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:54.334456 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f31f7-cf85-4227-ac68-1e3cb4c3777f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:54.347392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:54.349975 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:54.403054 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-f24f31f7-cf85-4227-ac68-1e3cb4c3777f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:54.405075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-f24f31f7-cf85-4227-ac68-1e3cb4c3777f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:54.466928 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca6e0c-7148-48bb-8f5a-ea0d1e7cd01c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:54.500905 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 693/1392] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:49:54.608748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:54.608748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0c3d9-0ed6-4f96-b0e7-2c90f6b94666 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:54.608748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0c3d9-0ed6-4f96-b0e7-2c90f6b94666 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:54.608748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0c3d9-0ed6-4f96-b0e7-2c90f6b94666 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:54.609269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0c3d9-0ed6-4f96-b0e7-2c90f6b94666 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:54.609269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-17e0c3d9-0ed6-4f96-b0e7-2c90f6b94666 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:54.687991 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0c3d9-0ed6-4f96-b0e7-2c90f6b94666 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:54.689611 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0c3d9-0ed6-4f96-b0e7-2c90f6b94666 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:54.709001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 700/1393] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:54.824519 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82155c-6d24-4e94-9eaa-4624ca6669eb None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:54.824801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82155c-6d24-4e94-9eaa-4624ca6669eb None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:54.825071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82155c-6d24-4e94-9eaa-4624ca6669eb None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:54.826420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 694/1394] 10.222.0.27 () {54 vars in 967 bytes} [Tue Aug 30 14:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:54.831267 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d79c20-4766-4a78-9264-f5dae58ab4f9 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:54.831557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d79c20-4766-4a78-9264-f5dae58ab4f9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:54.831834 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d79c20-4766-4a78-9264-f5dae58ab4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:54.909298 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-49d79c20-4766-4a78-9264-f5dae58ab4f9 None None] MFA Rules not processed for user `cfda7307e703427faaf599e32b91bc91`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:54.910473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-49d79c20-4766-4a78-9264-f5dae58ab4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:54.963564 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:54.992372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 701/1395] 10.222.0.27 () {58 vars in 1064 bytes} [Tue Aug 30 14:49:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:54.999981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82155c-6d24-4e94-9eaa-4624ca6669eb None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:55.001503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-fe82155c-6d24-4e94-9eaa-4624ca6669eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:55.108201 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:55.115295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82155c-6d24-4e94-9eaa-4624ca6669eb None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:55.197435 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ff719-0948-48f7-b84e-ee3352043a0b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfda7307e703427faaf599e32b91bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:55.197954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ff719-0948-48f7-b84e-ee3352043a0b service glance] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:55.198231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ff719-0948-48f7-b84e-ee3352043a0b service glance] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:55.198499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ff719-0948-48f7-b84e-ee3352043a0b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:55.200129 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a90ff719-0948-48f7-b84e-ee3352043a0b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:55.312897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ff719-0948-48f7-b84e-ee3352043a0b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:55.314391 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ff719-0948-48f7-b84e-ee3352043a0b service glance] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:55.334812 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 695/1396] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:56.268988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e554f-3317-45a8-a57d-47ded8ae5b78 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:56.269450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e554f-3317-45a8-a57d-47ded8ae5b78 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:56.269549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e554f-3317-45a8-a57d-47ded8ae5b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:56.337650 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-dc3e554f-3317-45a8-a57d-47ded8ae5b78 None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:56.338852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-dc3e554f-3317-45a8-a57d-47ded8ae5b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:56.425606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 702/1397] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 30 14:49:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:49:56.469012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fb1b2-f1d2-420d-816f-961bd7be695c None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:56.469494 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fb1b2-f1d2-420d-816f-961bd7be695c None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:56.469570 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fb1b2-f1d2-420d-816f-961bd7be695c None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:56.470939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 696/1398] 10.222.0.27 () {54 vars in 943 bytes} [Tue Aug 30 14:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:56.477266 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa99b0-92e8-4a95-9222-adb36a4cb941 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:56.478152 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa99b0-92e8-4a95-9222-adb36a4cb941 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:56.478487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa99b0-92e8-4a95-9222-adb36a4cb941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:56.554951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-46fa99b0-92e8-4a95-9222-adb36a4cb941 None None] MFA Rules not processed for user `bc23480d5aa94cad9e6599762ce01548`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:56.556760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-46fa99b0-92e8-4a95-9222-adb36a4cb941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:56.569473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:56.663510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 703/1399] 10.222.0.27 () {58 vars in 1040 bytes} [Tue Aug 30 14:49:56 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:49:56.675407 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fb1b2-f1d2-420d-816f-961bd7be695c None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:56.677159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-932fb1b2-f1d2-420d-816f-961bd7be695c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:56.700272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:56.815222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fb1b2-f1d2-420d-816f-961bd7be695c None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:56.909716 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da98992-885e-4029-848a-fe3e977a9a6d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:56.909997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da98992-885e-4029-848a-fe3e977a9a6d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:56.910295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da98992-885e-4029-848a-fe3e977a9a6d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:56.911678 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 704/1400] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:49:56.917323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40084f6e-7a61-45c6-80fc-8488b582e648 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:56.917606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40084f6e-7a61-45c6-80fc-8488b582e648 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:56.917877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40084f6e-7a61-45c6-80fc-8488b582e648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:56.917990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001cd8b-37e1-4ca1-a5ba-ff73a6e71a47 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:56.918575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001cd8b-37e1-4ca1-a5ba-ff73a6e71a47 service swift] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:56.918854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001cd8b-37e1-4ca1-a5ba-ff73a6e71a47 service swift] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:56.919145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001cd8b-37e1-4ca1-a5ba-ff73a6e71a47 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:56.920784 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-5001cd8b-37e1-4ca1-a5ba-ff73a6e71a47 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:56.928144 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:56.984677 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-40084f6e-7a61-45c6-80fc-8488b582e648 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:56.985911 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-40084f6e-7a61-45c6-80fc-8488b582e648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:57.005345 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001cd8b-37e1-4ca1-a5ba-ff73a6e71a47 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:57.006918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001cd8b-37e1-4ca1-a5ba-ff73a6e71a47 service swift] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:57.015624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 697/1401] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 14:49:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 14:49:57.062974 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:57.086407 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 705/1402] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:49:57.377700 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001cd8b-37e1-4ca1-a5ba-ff73a6e71a47 service swift] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:49:57.378896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-5001cd8b-37e1-4ca1-a5ba-ff73a6e71a47 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:57.494224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001cd8b-37e1-4ca1-a5ba-ff73a6e71a47 service swift] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:49:57.552084 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:49:57.617933 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a3996-c428-425f-8c41-eb44873ad1af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:49:57.619222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a3996-c428-425f-8c41-eb44873ad1af service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:57.619499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a3996-c428-425f-8c41-eb44873ad1af service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:57.619768 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a3996-c428-425f-8c41-eb44873ad1af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:57.622987 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e80a3996-c428-425f-8c41-eb44873ad1af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:57.727813 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a3996-c428-425f-8c41-eb44873ad1af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:49:57.729781 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a3996-c428-425f-8c41-eb44873ad1af service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:49:57.749419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 698/1403] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:49:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:49:59.857120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac423bd-c4aa-48b8-9983-37830d723647 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:59.857775 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac423bd-c4aa-48b8-9983-37830d723647 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:59.857775 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac423bd-c4aa-48b8-9983-37830d723647 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:59.859006 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 706/1404] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:49:59.864174 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801b45b-e91f-4447-bae7-c3716c3c3fee None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:49:59.864585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801b45b-e91f-4447-bae7-c3716c3c3fee None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:49:59.864685 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801b45b-e91f-4447-bae7-c3716c3c3fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:49:59.947948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-2801b45b-e91f-4447-bae7-c3716c3c3fee None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:49:59.951564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-2801b45b-e91f-4447-bae7-c3716c3c3fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:49:59.967685 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:00.061778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 699/1405] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:50:00.359946 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac423bd-c4aa-48b8-9983-37830d723647 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:00.361500 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-dac423bd-c4aa-48b8-9983-37830d723647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:00.527343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac423bd-c4aa-48b8-9983-37830d723647 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:00.588767 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:00.647761 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c31a2-5739-45dd-9024-41dc89193ec1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:00.648373 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c31a2-5739-45dd-9024-41dc89193ec1 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:00.648651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c31a2-5739-45dd-9024-41dc89193ec1 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:00.648922 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c31a2-5739-45dd-9024-41dc89193ec1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:00.651156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b92c31a2-5739-45dd-9024-41dc89193ec1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:00.755375 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c31a2-5739-45dd-9024-41dc89193ec1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:00.757107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c31a2-5739-45dd-9024-41dc89193ec1 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:00.778691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 707/1406] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:02.947087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894746b-27a3-496c-a3fa-9f335cbeb1de None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:02.947673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894746b-27a3-496c-a3fa-9f335cbeb1de None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:02.947673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894746b-27a3-496c-a3fa-9f335cbeb1de None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:02.948982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 700/1407] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:02.954231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bb171-d3e9-4d89-b55c-3d9cc383c100 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:02.954651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bb171-d3e9-4d89-b55c-3d9cc383c100 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:02.954803 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bb171-d3e9-4d89-b55c-3d9cc383c100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:03.024251 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f38bb171-d3e9-4d89-b55c-3d9cc383c100 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:03.025572 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f38bb171-d3e9-4d89-b55c-3d9cc383c100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:03.045740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:03.140093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 708/1408] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:03.429590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894746b-27a3-496c-a3fa-9f335cbeb1de None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:03.433328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b894746b-27a3-496c-a3fa-9f335cbeb1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:03.611087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:03.615742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894746b-27a3-496c-a3fa-9f335cbeb1de None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:03.750412 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64834651-9378-48f2-b06f-77f569190297 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:03.751678 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64834651-9378-48f2-b06f-77f569190297 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:03.752096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64834651-9378-48f2-b06f-77f569190297 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:03.752374 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64834651-9378-48f2-b06f-77f569190297 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:03.755889 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-64834651-9378-48f2-b06f-77f569190297 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:03.856849 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64834651-9378-48f2-b06f-77f569190297 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:03.858406 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64834651-9378-48f2-b06f-77f569190297 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:03.878680 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 701/1409] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:13.138198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=35 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:13.875997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:20.494720 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f600f0-dc59-48d9-9fb5-e3d0a21c1d34 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:20.495298 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f600f0-dc59-48d9-9fb5-e3d0a21c1d34 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:20.495298 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f600f0-dc59-48d9-9fb5-e3d0a21c1d34 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:20.496649 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 709/1410] 10.222.0.27 () {54 vars in 918 bytes} [Tue Aug 30 14:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:20.501927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a4815-a7ef-4162-939b-17c7952b48de None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:20.502330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a4815-a7ef-4162-939b-17c7952b48de None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:20.502475 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a4815-a7ef-4162-939b-17c7952b48de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:20.577968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-cf8a4815-a7ef-4162-939b-17c7952b48de None None] MFA Rules not processed for user `f4f11655cd51438e9652869f74ffa975`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:20.579120 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-cf8a4815-a7ef-4162-939b-17c7952b48de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:20.684776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 702/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Tue Aug 30 14:50:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:50:20.698850 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27693ff4-a90a-46ad-b423-a5a0edcaa431 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:20.699154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27693ff4-a90a-46ad-b423-a5a0edcaa431 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:20.699428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27693ff4-a90a-46ad-b423-a5a0edcaa431 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:20.700768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 710/1412] 10.222.0.27 () {54 vars in 918 bytes} [Tue Aug 30 14:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:20.705509 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975a0a8-5a81-449e-9873-eadaeddd535f None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:20.705796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975a0a8-5a81-449e-9873-eadaeddd535f None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:20.706070 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975a0a8-5a81-449e-9873-eadaeddd535f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:20.778713 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:20.781234 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-b975a0a8-5a81-449e-9873-eadaeddd535f None None] MFA Rules not processed for user `5b6e4c5bf2a54b28a55e5ae74b90ab4f`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:20.782442 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b975a0a8-5a81-449e-9873-eadaeddd535f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:20.896927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 703/1413] 10.222.0.27 () {58 vars in 1015 bytes} [Tue Aug 30 14:50:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:23.261523 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7422f0-3d9e-41c2-848b-216f595cb269 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:23.262078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7422f0-3d9e-41c2-848b-216f595cb269 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:23.262078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7422f0-3d9e-41c2-848b-216f595cb269 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:23.263224 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:23.268492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a60d3-76a7-483a-bd30-73b209e96cce None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:23.268896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a60d3-76a7-483a-bd30-73b209e96cce None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:23.269042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a60d3-76a7-483a-bd30-73b209e96cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:23.334353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-693a60d3-76a7-483a-bd30-73b209e96cce None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:23.335573 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-693a60d3-76a7-483a-bd30-73b209e96cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:23.389951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:23.431678 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:50:23.723020 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7422f0-3d9e-41c2-848b-216f595cb269 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:23.724604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4f7422f0-3d9e-41c2-848b-216f595cb269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:23.868737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7422f0-3d9e-41c2-848b-216f595cb269 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:23.939426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:23.978994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfa80d-a578-4da7-b5fa-cd0f4ad308bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:23.979641 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfa80d-a578-4da7-b5fa-cd0f4ad308bd service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:23.979896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfa80d-a578-4da7-b5fa-cd0f4ad308bd service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:23.980234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfa80d-a578-4da7-b5fa-cd0f4ad308bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:23.981805 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-59cfa80d-a578-4da7-b5fa-cd0f4ad308bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:24.069146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfa80d-a578-4da7-b5fa-cd0f4ad308bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:24.070812 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfa80d-a578-4da7-b5fa-cd0f4ad308bd service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:24.089915 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 712/1416] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:24.969689 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1da7e-5527-4993-8777-f279c11b133e None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:24.970227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1da7e-5527-4993-8777-f279c11b133e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:24.970227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1da7e-5527-4993-8777-f279c11b133e None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:24.971629 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:24.976512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b054c8b-afdd-4611-a82d-d4e60a64e660 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:24.976923 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b054c8b-afdd-4611-a82d-d4e60a64e660 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:24.977052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b054c8b-afdd-4611-a82d-d4e60a64e660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:25.051812 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-7b054c8b-afdd-4611-a82d-d4e60a64e660 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:25.053004 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7b054c8b-afdd-4611-a82d-d4e60a64e660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:25.080157 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:25.150088 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:25.298728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1da7e-5527-4993-8777-f279c11b133e None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:25.300273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-abc1da7e-5527-4993-8777-f279c11b133e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:25.443404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1da7e-5527-4993-8777-f279c11b133e None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:25.483323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:25.555684 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbefb347-e391-4911-b855-8d9548f4e19e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:25.556208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefb347-e391-4911-b855-8d9548f4e19e service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:25.556482 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefb347-e391-4911-b855-8d9548f4e19e service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:25.556752 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefb347-e391-4911-b855-8d9548f4e19e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:25.558334 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-fbefb347-e391-4911-b855-8d9548f4e19e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:25.644717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbefb347-e391-4911-b855-8d9548f4e19e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:25.646286 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbefb347-e391-4911-b855-8d9548f4e19e service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:25.664236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 706/1419] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:26.552411 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a08d0f-9bfc-48d6-bd83-da5b03534e16 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:26.552937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a08d0f-9bfc-48d6-bd83-da5b03534e16 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:26.552937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a08d0f-9bfc-48d6-bd83-da5b03534e16 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:26.554258 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:26.559065 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1efd6-6a50-40c5-a1d7-70352ccfe9e1 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:26.559468 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1efd6-6a50-40c5-a1d7-70352ccfe9e1 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:26.559643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1efd6-6a50-40c5-a1d7-70352ccfe9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:26.614013 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:26.636472 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-30e1efd6-6a50-40c5-a1d7-70352ccfe9e1 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:26.637734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-30e1efd6-6a50-40c5-a1d7-70352ccfe9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:26.732065 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:26.883269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a08d0f-9bfc-48d6-bd83-da5b03534e16 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:26.884724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d8a08d0f-9bfc-48d6-bd83-da5b03534e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:27.029658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a08d0f-9bfc-48d6-bd83-da5b03534e16 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:27.042223 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:27.142536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4ec3e-335a-4455-8395-a30dc01b16cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:27.143057 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4ec3e-335a-4455-8395-a30dc01b16cf service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:27.143358 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4ec3e-335a-4455-8395-a30dc01b16cf service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:27.143621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4ec3e-335a-4455-8395-a30dc01b16cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:27.145195 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b0a4ec3e-335a-4455-8395-a30dc01b16cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:27.232555 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a4ec3e-335a-4455-8395-a30dc01b16cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:27.234185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a4ec3e-335a-4455-8395-a30dc01b16cf service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:27.252337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 715/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:28.107688 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83eba1-9a8c-4042-adff-cf43a8cfee39 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:28.108258 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83eba1-9a8c-4042-adff-cf43a8cfee39 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:28.108258 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83eba1-9a8c-4042-adff-cf43a8cfee39 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:28.109599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:28.114547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41ca84-0681-40d5-988f-73dac5766879 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:28.114961 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41ca84-0681-40d5-988f-73dac5766879 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:28.115090 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41ca84-0681-40d5-988f-73dac5766879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:28.136248 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:28.192353 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6f41ca84-0681-40d5-988f-73dac5766879 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:28.193558 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6f41ca84-0681-40d5-988f-73dac5766879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:28.288954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:28.440311 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83eba1-9a8c-4042-adff-cf43a8cfee39 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:28.441844 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-fc83eba1-9a8c-4042-adff-cf43a8cfee39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:28.553095 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:28.589021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83eba1-9a8c-4042-adff-cf43a8cfee39 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:28.701348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9789ae-2d5e-4c8d-aa2a-3daa2be780d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:28.701868 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9789ae-2d5e-4c8d-aa2a-3daa2be780d2 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:28.702142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9789ae-2d5e-4c8d-aa2a-3daa2be780d2 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:28.702409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9789ae-2d5e-4c8d-aa2a-3daa2be780d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:28.704035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ce9789ae-2d5e-4c8d-aa2a-3daa2be780d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:28.779230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:28.792802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9789ae-2d5e-4c8d-aa2a-3daa2be780d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:28.794431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9789ae-2d5e-4c8d-aa2a-3daa2be780d2 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:28.813086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 709/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:29.681537 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51c430-45b5-472b-9134-34d17f937533 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:29.682043 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51c430-45b5-472b-9134-34d17f937533 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:29.682043 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51c430-45b5-472b-9134-34d17f937533 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:29.683408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:29.688435 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84292315-a072-469a-b393-d10c89e34ca2 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:29.688833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84292315-a072-469a-b393-d10c89e34ca2 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:29.688981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84292315-a072-469a-b393-d10c89e34ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:29.766453 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-84292315-a072-469a-b393-d10c89e34ca2 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:29.767701 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-84292315-a072-469a-b393-d10c89e34ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:29.862716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:30.014175 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51c430-45b5-472b-9134-34d17f937533 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:30.015739 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ed51c430-45b5-472b-9134-34d17f937533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:30.222614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:30.283539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51c430-45b5-472b-9134-34d17f937533 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:30.393966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23172f4d-5659-4fc2-9371-49ac104c25de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:30.394566 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23172f4d-5659-4fc2-9371-49ac104c25de service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:30.394845 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23172f4d-5659-4fc2-9371-49ac104c25de service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:30.395133 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23172f4d-5659-4fc2-9371-49ac104c25de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:30.396725 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-23172f4d-5659-4fc2-9371-49ac104c25de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:30.443866 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:30.483883 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23172f4d-5659-4fc2-9371-49ac104c25de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:30.485460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23172f4d-5659-4fc2-9371-49ac104c25de service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:30.503615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 718/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:31.397691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cbfb6-b1af-465f-8d42-8de9be43b2fa None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:31.398769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cbfb6-b1af-465f-8d42-8de9be43b2fa None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:31.398769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cbfb6-b1af-465f-8d42-8de9be43b2fa None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:31.400170 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:50:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:31.405274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1c1ce-3f99-4b89-8d3e-a3722ff82f6c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:31.405274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1c1ce-3f99-4b89-8d3e-a3722ff82f6c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:31.405740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1c1ce-3f99-4b89-8d3e-a3722ff82f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:31.480101 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b2e1c1ce-3f99-4b89-8d3e-a3722ff82f6c None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:31.481317 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b2e1c1ce-3f99-4b89-8d3e-a3722ff82f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:31.577869 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:31.729837 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cbfb6-b1af-465f-8d42-8de9be43b2fa None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:31.731414 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7b9cbfb6-b1af-465f-8d42-8de9be43b2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:31.780430 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:31.875303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cbfb6-b1af-465f-8d42-8de9be43b2fa None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:31.988140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c70aa-5ca9-4142-8e12-ba7a1cd35ce6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:31.988795 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c70aa-5ca9-4142-8e12-ba7a1cd35ce6 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:31.989158 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c70aa-5ca9-4142-8e12-ba7a1cd35ce6 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:31.989427 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c70aa-5ca9-4142-8e12-ba7a1cd35ce6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:31.991183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a70c70aa-5ca9-4142-8e12-ba7a1cd35ce6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:31.998710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:32.081262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c70aa-5ca9-4142-8e12-ba7a1cd35ce6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:32.082847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c70aa-5ca9-4142-8e12-ba7a1cd35ce6 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:32.100347 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 712/1431] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:32.954254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25debe-2c79-4ddf-a488-26d435510e0f None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:32.954751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25debe-2c79-4ddf-a488-26d435510e0f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:32.954854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25debe-2c79-4ddf-a488-26d435510e0f None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:32.956157 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:32.961139 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b7e43-c91c-4424-817a-5954ed9114da None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:32.961565 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b7e43-c91c-4424-817a-5954ed9114da None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:32.961668 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b7e43-c91c-4424-817a-5954ed9114da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:33.036901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-ae5b7e43-c91c-4424-817a-5954ed9114da None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:33.038164 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ae5b7e43-c91c-4424-817a-5954ed9114da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:33.104638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:33.133103 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:33.283335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25debe-2c79-4ddf-a488-26d435510e0f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:33.284835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4a25debe-2c79-4ddf-a488-26d435510e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:33.302814 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:33.428364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25debe-2c79-4ddf-a488-26d435510e0f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:33.513759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:33.540049 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c93dd9-b3c8-4e48-9cea-2fbbc1ce58f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:33.540584 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c93dd9-b3c8-4e48-9cea-2fbbc1ce58f8 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:33.540858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c93dd9-b3c8-4e48-9cea-2fbbc1ce58f8 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:33.541125 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c93dd9-b3c8-4e48-9cea-2fbbc1ce58f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:33.542709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d1c93dd9-b3c8-4e48-9cea-2fbbc1ce58f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:33.629362 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c93dd9-b3c8-4e48-9cea-2fbbc1ce58f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:33.630943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c93dd9-b3c8-4e48-9cea-2fbbc1ce58f8 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:33.648854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 721/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:34.528304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3a317-1454-4d2d-bd51-d7cf09833b43 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:34.528827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3a317-1454-4d2d-bd51-d7cf09833b43 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:34.528827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3a317-1454-4d2d-bd51-d7cf09833b43 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:34.530106 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:34.534954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522f675-dd83-4e90-b269-1b9fa892f2ef None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:34.535390 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522f675-dd83-4e90-b269-1b9fa892f2ef None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:34.535545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522f675-dd83-4e90-b269-1b9fa892f2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:34.609966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a522f675-dd83-4e90-b269-1b9fa892f2ef None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:34.611190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a522f675-dd83-4e90-b269-1b9fa892f2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:34.652780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:34.706820 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:34.856136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3a317-1454-4d2d-bd51-d7cf09833b43 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:34.857587 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6eb3a317-1454-4d2d-bd51-d7cf09833b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:35.003958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3a317-1454-4d2d-bd51-d7cf09833b43 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:35.058198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:35.116550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f34d013-c62d-494d-a80a-ec6440d1457a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:35.117071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34d013-c62d-494d-a80a-ec6440d1457a service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:35.117345 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34d013-c62d-494d-a80a-ec6440d1457a service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:35.117611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34d013-c62d-494d-a80a-ec6440d1457a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:35.119220 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8f34d013-c62d-494d-a80a-ec6440d1457a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:35.205722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f34d013-c62d-494d-a80a-ec6440d1457a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:35.207307 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f34d013-c62d-494d-a80a-ec6440d1457a service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:35.225415 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 715/1437] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:36.085625 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865ac9d-ebc9-4c8b-ad49-ce4d78b00305 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:36.086199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865ac9d-ebc9-4c8b-ad49-ce4d78b00305 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:36.086199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865ac9d-ebc9-4c8b-ad49-ce4d78b00305 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:36.087556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:36.092462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffa74f-22ec-45e4-bcb3-d403081147bc None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:36.092907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffa74f-22ec-45e4-bcb3-d403081147bc None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:36.093016 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffa74f-22ec-45e4-bcb3-d403081147bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:36.169666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-b1ffa74f-22ec-45e4-bcb3-d403081147bc None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:36.170879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b1ffa74f-22ec-45e4-bcb3-d403081147bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:36.172953 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:36.268432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:36.418977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865ac9d-ebc9-4c8b-ad49-ce4d78b00305 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:36.420434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a865ac9d-ebc9-4c8b-ad49-ce4d78b00305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:36.562196 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865ac9d-ebc9-4c8b-ad49-ce4d78b00305 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:36.589201 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:36.673834 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f415f22-5ddc-447c-a43a-f61a2baa5bd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:36.674359 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f415f22-5ddc-447c-a43a-f61a2baa5bd6 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:36.674635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f415f22-5ddc-447c-a43a-f61a2baa5bd6 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:36.674905 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f415f22-5ddc-447c-a43a-f61a2baa5bd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:36.676527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2f415f22-5ddc-447c-a43a-f61a2baa5bd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:36.763075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f415f22-5ddc-447c-a43a-f61a2baa5bd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:36.764668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f415f22-5ddc-447c-a43a-f61a2baa5bd6 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:36.782632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 724/1440] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:38.582744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21ec3b-aa3c-47e5-bd5a-1123837e83a2 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:38.583329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21ec3b-aa3c-47e5-bd5a-1123837e83a2 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:38.583329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21ec3b-aa3c-47e5-bd5a-1123837e83a2 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:38.584650 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:38.589896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58941ef7-9931-4a65-8ec4-70998e509cb0 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:38.590332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58941ef7-9931-4a65-8ec4-70998e509cb0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:38.590441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58941ef7-9931-4a65-8ec4-70998e509cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:38.633296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:38.661971 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-58941ef7-9931-4a65-8ec4-70998e509cb0 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:38.663742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-58941ef7-9931-4a65-8ec4-70998e509cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:38.764187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:39.049286 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21ec3b-aa3c-47e5-bd5a-1123837e83a2 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:39.050824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-aa21ec3b-aa3c-47e5-bd5a-1123837e83a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:39.153362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21ec3b-aa3c-47e5-bd5a-1123837e83a2 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:39.184292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:39.225328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229551eb-da3a-4f90-9ed1-583686805aa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:39.225929 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229551eb-da3a-4f90-9ed1-583686805aa7 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:39.226206 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229551eb-da3a-4f90-9ed1-583686805aa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:39.226473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229551eb-da3a-4f90-9ed1-583686805aa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:39.228100 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-229551eb-da3a-4f90-9ed1-583686805aa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:39.336643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229551eb-da3a-4f90-9ed1-583686805aa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:39.338000 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229551eb-da3a-4f90-9ed1-583686805aa7 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:39.356408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 718/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:41.400859 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20693626-f54f-48bb-8e82-8a753e498009 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:41.401404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20693626-f54f-48bb-8e82-8a753e498009 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:41.401404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20693626-f54f-48bb-8e82-8a753e498009 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:41.402765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:41.408106 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11795ed2-bf6e-4b6f-aba5-7ee8db8b6a73 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:41.408532 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11795ed2-bf6e-4b6f-aba5-7ee8db8b6a73 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:41.408638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11795ed2-bf6e-4b6f-aba5-7ee8db8b6a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:41.474733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-11795ed2-bf6e-4b6f-aba5-7ee8db8b6a73 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:41.475991 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-11795ed2-bf6e-4b6f-aba5-7ee8db8b6a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:41.488870 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:41.578601 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:50:41.858198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20693626-f54f-48bb-8e82-8a753e498009 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:41.859755 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-20693626-f54f-48bb-8e82-8a753e498009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:41.960797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20693626-f54f-48bb-8e82-8a753e498009 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:41.972465 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:42.032539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237e344-727e-4a16-ac27-7ff930342763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:42.033143 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237e344-727e-4a16-ac27-7ff930342763 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:42.033421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237e344-727e-4a16-ac27-7ff930342763 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:42.033718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237e344-727e-4a16-ac27-7ff930342763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:42.035471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7237e344-727e-4a16-ac27-7ff930342763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:42.141718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237e344-727e-4a16-ac27-7ff930342763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:42.143075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237e344-727e-4a16-ac27-7ff930342763 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:42.160924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 727/1446] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:44.413602 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b7817-8d05-458d-b6a4-31891053c57b None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:44.414155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b7817-8d05-458d-b6a4-31891053c57b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:44.414155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b7817-8d05-458d-b6a4-31891053c57b None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:44.415516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:50:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:44.421603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73244fcd-c9dc-48c0-ba42-b0501149559b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:44.422031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73244fcd-c9dc-48c0-ba42-b0501149559b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:44.422149 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73244fcd-c9dc-48c0-ba42-b0501149559b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:44.470623 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:44.488884 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-73244fcd-c9dc-48c0-ba42-b0501149559b None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:44.490023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-73244fcd-c9dc-48c0-ba42-b0501149559b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:44.584758 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:44.868336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b7817-8d05-458d-b6a4-31891053c57b None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:44.869876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-222b7817-8d05-458d-b6a4-31891053c57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:44.971216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b7817-8d05-458d-b6a4-31891053c57b None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:45.010625 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:45.043375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fe912-9677-4938-be74-181104f368ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:45.043993 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fe912-9677-4938-be74-181104f368ca service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:45.044268 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fe912-9677-4938-be74-181104f368ca service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:45.044539 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fe912-9677-4938-be74-181104f368ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:45.046119 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6f5fe912-9677-4938-be74-181104f368ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:45.159883 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fe912-9677-4938-be74-181104f368ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:45.161225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fe912-9677-4938-be74-181104f368ca service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:45.179417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 721/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:47.240012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d1769-2108-40f4-a504-3c8c2da99f40 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:47.240588 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d1769-2108-40f4-a504-3c8c2da99f40 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:47.240588 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d1769-2108-40f4-a504-3c8c2da99f40 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:47.242388 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:47.247657 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0561b01-6b26-466d-a3b0-db03726d8e0e None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:47.248085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0561b01-6b26-466d-a3b0-db03726d8e0e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:47.248194 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0561b01-6b26-466d-a3b0-db03726d8e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:47.314628 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-f0561b01-6b26-466d-a3b0-db03726d8e0e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:47.315841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-f0561b01-6b26-466d-a3b0-db03726d8e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:47.335655 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:47.410172 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:47.699030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d1769-2108-40f4-a504-3c8c2da99f40 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:47.700593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-092d1769-2108-40f4-a504-3c8c2da99f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:47.803911 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d1769-2108-40f4-a504-3c8c2da99f40 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:47.823047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:47.876159 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb650d-38cb-4f49-9c6b-96d041ab33a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:47.876755 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb650d-38cb-4f49-9c6b-96d041ab33a9 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:47.877026 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb650d-38cb-4f49-9c6b-96d041ab33a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:47.877314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb650d-38cb-4f49-9c6b-96d041ab33a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:47.878884 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-32bb650d-38cb-4f49-9c6b-96d041ab33a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:47.984581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb650d-38cb-4f49-9c6b-96d041ab33a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:47.986010 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb650d-38cb-4f49-9c6b-96d041ab33a9 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:48.003659 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 730/1452] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:49.948459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98397cdd-09aa-45b2-a427-3f772c981ddf None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:49.949023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98397cdd-09aa-45b2-a427-3f772c981ddf None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:49.949140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98397cdd-09aa-45b2-a427-3f772c981ddf None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:49.950449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:49.955722 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef305d7-9532-4650-8e64-7f3f001803b4 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:49.956137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef305d7-9532-4650-8e64-7f3f001803b4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:49.956270 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef305d7-9532-4650-8e64-7f3f001803b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:50.019028 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:50.021710 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bef305d7-9532-4650-8e64-7f3f001803b4 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:50.023078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bef305d7-9532-4650-8e64-7f3f001803b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:50.119425 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:50.401305 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98397cdd-09aa-45b2-a427-3f772c981ddf None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:50.402876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-98397cdd-09aa-45b2-a427-3f772c981ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:50.507320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98397cdd-09aa-45b2-a427-3f772c981ddf None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:50.555825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:50.579905 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475ba59-589f-4b1a-b1dc-ae6b6db13cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:50.580495 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475ba59-589f-4b1a-b1dc-ae6b6db13cac service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:50.580767 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475ba59-589f-4b1a-b1dc-ae6b6db13cac service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:50.581036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475ba59-589f-4b1a-b1dc-ae6b6db13cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:50.582608 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-2475ba59-589f-4b1a-b1dc-ae6b6db13cac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:50.688945 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475ba59-589f-4b1a-b1dc-ae6b6db13cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:50.690352 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475ba59-589f-4b1a-b1dc-ae6b6db13cac service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:50.708207 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 724/1455] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:52.664861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb17b39-2335-4e1c-94ee-025ff009512e None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:52.665379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb17b39-2335-4e1c-94ee-025ff009512e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:52.665379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb17b39-2335-4e1c-94ee-025ff009512e None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:52.666703 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:52.672021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439dc677-024e-478e-9fff-4d756fc22cc4 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:52.672455 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439dc677-024e-478e-9fff-4d756fc22cc4 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:52.672574 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439dc677-024e-478e-9fff-4d756fc22cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:52.738254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-439dc677-024e-478e-9fff-4d756fc22cc4 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:52.739502 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-439dc677-024e-478e-9fff-4d756fc22cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:52.769451 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:52.836471 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:53.125840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb17b39-2335-4e1c-94ee-025ff009512e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:53.127434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5fb17b39-2335-4e1c-94ee-025ff009512e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:53.230261 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb17b39-2335-4e1c-94ee-025ff009512e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:53.259603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:53.306047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942ac7a-f036-4698-9fda-8452072138a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:53.306586 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942ac7a-f036-4698-9fda-8452072138a5 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:53.306865 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942ac7a-f036-4698-9fda-8452072138a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:53.307159 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942ac7a-f036-4698-9fda-8452072138a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:53.308750 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1942ac7a-f036-4698-9fda-8452072138a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:53.413636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942ac7a-f036-4698-9fda-8452072138a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:53.415065 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942ac7a-f036-4698-9fda-8452072138a5 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:53.433969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 733/1458] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:55.449001 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443905a6-1e24-4b79-a2b8-ed7550e54a11 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:55.449471 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443905a6-1e24-4b79-a2b8-ed7550e54a11 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:55.449585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443905a6-1e24-4b79-a2b8-ed7550e54a11 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:55.450861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:55.456265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcb7a7-4a99-4013-b8bf-3fa79039a426 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:55.456676 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcb7a7-4a99-4013-b8bf-3fa79039a426 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:55.456809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcb7a7-4a99-4013-b8bf-3fa79039a426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:55.522865 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-9cbcb7a7-4a99-4013-b8bf-3fa79039a426 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:55.524018 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9cbcb7a7-4a99-4013-b8bf-3fa79039a426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:55.532125 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:55.619646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:50:55.903939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443905a6-1e24-4b79-a2b8-ed7550e54a11 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:55.905520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-443905a6-1e24-4b79-a2b8-ed7550e54a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:56.007933 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443905a6-1e24-4b79-a2b8-ed7550e54a11 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:56.062890 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:56.081916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57169d7f-0055-4449-8fa3-a0a6877f2716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:56.082536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57169d7f-0055-4449-8fa3-a0a6877f2716 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:56.083024 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57169d7f-0055-4449-8fa3-a0a6877f2716 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:56.083506 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57169d7f-0055-4449-8fa3-a0a6877f2716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:56.085436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-57169d7f-0055-4449-8fa3-a0a6877f2716 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:56.200075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57169d7f-0055-4449-8fa3-a0a6877f2716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:56.201509 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57169d7f-0055-4449-8fa3-a0a6877f2716 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:56.219837 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 727/1461] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:58.141781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4403c-db8d-44b5-be0d-2167035efa83 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:58.142279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4403c-db8d-44b5-be0d-2167035efa83 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:58.142393 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4403c-db8d-44b5-be0d-2167035efa83 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:58.143692 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:50:58.149544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4b37d-1562-4b41-a1bb-87a482dc2943 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:58.149984 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4b37d-1562-4b41-a1bb-87a482dc2943 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:58.150080 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4b37d-1562-4b41-a1bb-87a482dc2943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:58.217346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-31f4b37d-1562-4b41-a1bb-87a482dc2943 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:50:58.218553 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-31f4b37d-1562-4b41-a1bb-87a482dc2943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:58.257665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:58.317048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:50:58.602711 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4403c-db8d-44b5-be0d-2167035efa83 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:58.604280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-33e4403c-db8d-44b5-be0d-2167035efa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:58.706311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4403c-db8d-44b5-be0d-2167035efa83 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:58.743230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:58.779393 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b8eae-99f0-4288-acc1-9b9cc8d2b829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:58.779994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b8eae-99f0-4288-acc1-9b9cc8d2b829 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:58.780272 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b8eae-99f0-4288-acc1-9b9cc8d2b829 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:58.780540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b8eae-99f0-4288-acc1-9b9cc8d2b829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:58.782127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-682b8eae-99f0-4288-acc1-9b9cc8d2b829 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:58.889397 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b8eae-99f0-4288-acc1-9b9cc8d2b829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:58.890738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b8eae-99f0-4288-acc1-9b9cc8d2b829 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:58.908590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 736/1464] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:51:00.805928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b91ec-0322-441a-8ae9-53a84382b9fe None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:00.806506 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b91ec-0322-441a-8ae9-53a84382b9fe None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:00.806506 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b91ec-0322-441a-8ae9-53a84382b9fe None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:00.807881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:51:00.813075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112156a6-0108-443d-a740-af04c5375a24 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:00.813499 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112156a6-0108-443d-a740-af04c5375a24 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:00.813638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112156a6-0108-443d-a740-af04c5375a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:00.882970 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-112156a6-0108-443d-a740-af04c5375a24 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:51:00.884218 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-112156a6-0108-443d-a740-af04c5375a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:00.902227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:00.980056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:51:01.259550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b91ec-0322-441a-8ae9-53a84382b9fe None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:51:01.261121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9c2b91ec-0322-441a-8ae9-53a84382b9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:01.363465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b91ec-0322-441a-8ae9-53a84382b9fe None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:51:01.428494 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:01.435245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2e30d-e2f1-4b3d-b15c-c9f1219c8d79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:51:01.435842 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2e30d-e2f1-4b3d-b15c-c9f1219c8d79 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:01.436116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2e30d-e2f1-4b3d-b15c-c9f1219c8d79 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:01.436383 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2e30d-e2f1-4b3d-b15c-c9f1219c8d79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:01.437960 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-88d2e30d-e2f1-4b3d-b15c-c9f1219c8d79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:01.545680 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2e30d-e2f1-4b3d-b15c-c9f1219c8d79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:51:01.547095 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2e30d-e2f1-4b3d-b15c-c9f1219c8d79 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:51:01.565528 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 730/1467] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:51:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:51:03.464407 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed5cb8-f63a-4d39-8a90-8fdb58017104 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:03.464928 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed5cb8-f63a-4d39-8a90-8fdb58017104 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:03.464928 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed5cb8-f63a-4d39-8a90-8fdb58017104 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:03.466284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:51:03.471462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517b431-7663-48ef-9c3d-33339073a0b3 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:03.471916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517b431-7663-48ef-9c3d-33339073a0b3 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:03.472019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517b431-7663-48ef-9c3d-33339073a0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:03.539140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-7517b431-7663-48ef-9c3d-33339073a0b3 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:51:03.540233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7517b431-7663-48ef-9c3d-33339073a0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:03.591321 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:03.639024 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:51:03.923167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed5cb8-f63a-4d39-8a90-8fdb58017104 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:51:03.924699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-46ed5cb8-f63a-4d39-8a90-8fdb58017104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:04.026417 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed5cb8-f63a-4d39-8a90-8fdb58017104 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:51:04.073400 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:04.098416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae6a7e-dec0-44fd-b0de-dab57781c8ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:51:04.099016 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae6a7e-dec0-44fd-b0de-dab57781c8ee service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:04.099815 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae6a7e-dec0-44fd-b0de-dab57781c8ee service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:04.100083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae6a7e-dec0-44fd-b0de-dab57781c8ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:04.101678 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-dfae6a7e-dec0-44fd-b0de-dab57781c8ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:04.208535 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae6a7e-dec0-44fd-b0de-dab57781c8ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:51:04.210391 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae6a7e-dec0-44fd-b0de-dab57781c8ee service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:51:04.229002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 739/1470] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:51:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:51:13.646986 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=22 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:14.226804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=66 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:23.583087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09408b-7a9e-4226-b54c-28862c44b907 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:23.583633 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09408b-7a9e-4226-b54c-28862c44b907 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:23.583633 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09408b-7a9e-4226-b54c-28862c44b907 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:23.584961 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 732/1471] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:51:23.590218 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27f64f-aa3a-4921-8fd2-d1e3b05eca60 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:23.590636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27f64f-aa3a-4921-8fd2-d1e3b05eca60 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:23.590766 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27f64f-aa3a-4921-8fd2-d1e3b05eca60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:23.655505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-df27f64f-aa3a-4921-8fd2-d1e3b05eca60 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:51:23.656695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-df27f64f-aa3a-4921-8fd2-d1e3b05eca60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:23.749716 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 740/1472] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:51:24.042071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d261aa-48af-4783-900e-493ed6de8fe9 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:24.042359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d261aa-48af-4783-900e-493ed6de8fe9 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:24.042637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d261aa-48af-4783-900e-493ed6de8fe9 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:24.044016 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 733/1473] 10.222.0.27 () {54 vars in 967 bytes} [Tue Aug 30 14:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:51:24.048630 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2ca30-59cd-4d6d-ae5e-22f992c5e84e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:24.048916 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2ca30-59cd-4d6d-ae5e-22f992c5e84e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:24.049184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2ca30-59cd-4d6d-ae5e-22f992c5e84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:24.122702 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5ba2ca30-59cd-4d6d-ae5e-22f992c5e84e None None] MFA Rules not processed for user `cfda7307e703427faaf599e32b91bc91`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:51:24.124183 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5ba2ca30-59cd-4d6d-ae5e-22f992c5e84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:24.144724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:24.200071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 741/1474] 10.222.0.27 () {58 vars in 1064 bytes} [Tue Aug 30 14:51:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:51:24.207497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d261aa-48af-4783-900e-493ed6de8fe9 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:51:24.209598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-43d261aa-48af-4783-900e-493ed6de8fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:24.315268 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d261aa-48af-4783-900e-493ed6de8fe9 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:51:24.387579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529e401-c7bc-4de9-9368-f9a599802ef1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfda7307e703427faaf599e32b91bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:51:24.388170 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529e401-c7bc-4de9-9368-f9a599802ef1 service glance] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:24.388446 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529e401-c7bc-4de9-9368-f9a599802ef1 service glance] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:24.388714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529e401-c7bc-4de9-9368-f9a599802ef1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:24.390290 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a529e401-c7bc-4de9-9368-f9a599802ef1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:24.446986 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:24.501302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529e401-c7bc-4de9-9368-f9a599802ef1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:51:24.502728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529e401-c7bc-4de9-9368-f9a599802ef1 service glance] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:51:24.521361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 734/1475] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:51:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:51:26.510500 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58385e-5625-40e7-9d64-b1d9a47ffda4 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:26.511028 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58385e-5625-40e7-9d64-b1d9a47ffda4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:26.511232 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58385e-5625-40e7-9d64-b1d9a47ffda4 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:26.512510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 742/1476] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:51:26.517932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad721261-af51-4937-832a-26fd4c9c77d8 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:26.518346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad721261-af51-4937-832a-26fd4c9c77d8 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:26.518486 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad721261-af51-4937-832a-26fd4c9c77d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:26.584331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-ad721261-af51-4937-832a-26fd4c9c77d8 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:51:26.585531 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ad721261-af51-4937-832a-26fd4c9c77d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:26.682043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 735/1477] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:51:26.954904 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58385e-5625-40e7-9d64-b1d9a47ffda4 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:51:26.956427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-dd58385e-5625-40e7-9d64-b1d9a47ffda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:27.082284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58385e-5625-40e7-9d64-b1d9a47ffda4 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:51:27.131171 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:27.174886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d6fd0-29df-40e8-8a1a-f861131d2321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:51:27.175572 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d6fd0-29df-40e8-8a1a-f861131d2321 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:27.175852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d6fd0-29df-40e8-8a1a-f861131d2321 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:27.176152 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d6fd0-29df-40e8-8a1a-f861131d2321 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:27.177409 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d6fd0-29df-40e8-8a1a-f861131d2321 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:51:27.179168 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: WARNING py.warnings [None req-124d6fd0-29df-40e8-8a1a-f861131d2321 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:51:27.179168 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: warnings.warn(msg) Aug 30 14:51:27.179168 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: Aug 30 14:51:27.180340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d6fd0-29df-40e8-8a1a-f861131d2321 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:51:27.199051 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 743/1478] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Aug 30 14:51:26 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:51:29.105852 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7064d4f-9c6d-4f6e-b57d-5f3b8a06a4e1 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:29.122400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7064d4f-9c6d-4f6e-b57d-5f3b8a06a4e1 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:29.122400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7064d4f-9c6d-4f6e-b57d-5f3b8a06a4e1 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:29.122400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 736/1479] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:51:29.122686 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3ab1f-2f68-4ed8-87f5-1af0d84adf95 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:29.122686 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3ab1f-2f68-4ed8-87f5-1af0d84adf95 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:29.122686 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3ab1f-2f68-4ed8-87f5-1af0d84adf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:29.179952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-37b3ab1f-2f68-4ed8-87f5-1af0d84adf95 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:51:29.181167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-37b3ab1f-2f68-4ed8-87f5-1af0d84adf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:29.277838 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 744/1480] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:51:29.574474 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7064d4f-9c6d-4f6e-b57d-5f3b8a06a4e1 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:51:29.576022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a7064d4f-9c6d-4f6e-b57d-5f3b8a06a4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:29.586459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:29.721598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7064d4f-9c6d-4f6e-b57d-5f3b8a06a4e1 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:51:29.800820 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:29.833469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca0a65-b92f-41cc-9482-30b682546da1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:51:29.834060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca0a65-b92f-41cc-9482-30b682546da1 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:29.834329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca0a65-b92f-41cc-9482-30b682546da1 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:29.834594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca0a65-b92f-41cc-9482-30b682546da1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:29.836198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-96ca0a65-b92f-41cc-9482-30b682546da1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:29.924269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca0a65-b92f-41cc-9482-30b682546da1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:51:29.925855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca0a65-b92f-41cc-9482-30b682546da1 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:51:29.943704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 737/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:51:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:51:32.029838 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23802a1-60a6-4dc5-8a23-2310704ae654 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:32.030406 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23802a1-60a6-4dc5-8a23-2310704ae654 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:32.030406 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23802a1-60a6-4dc5-8a23-2310704ae654 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:32.031764 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 745/1482] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:51:32.037104 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1da806-4bdc-4dee-91a5-24a1868624d1 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:32.037512 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1da806-4bdc-4dee-91a5-24a1868624d1 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:32.037652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1da806-4bdc-4dee-91a5-24a1868624d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:32.103750 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-3f1da806-4bdc-4dee-91a5-24a1868624d1 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:51:32.104978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-3f1da806-4bdc-4dee-91a5-24a1868624d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:32.151508 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:32.202095 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 738/1483] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:51:32.496854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23802a1-60a6-4dc5-8a23-2310704ae654 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:51:32.498640 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b23802a1-60a6-4dc5-8a23-2310704ae654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:32.521296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:32.599191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23802a1-60a6-4dc5-8a23-2310704ae654 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:51:32.671964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ed050-6ab2-4252-862f-5122e07cedba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:51:32.672568 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ed050-6ab2-4252-862f-5122e07cedba service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:32.672853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ed050-6ab2-4252-862f-5122e07cedba service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:32.673130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ed050-6ab2-4252-862f-5122e07cedba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:32.674814 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-273ed050-6ab2-4252-862f-5122e07cedba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:32.748500 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:32.782580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ed050-6ab2-4252-862f-5122e07cedba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:51:32.784032 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ed050-6ab2-4252-862f-5122e07cedba service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:51:32.802884 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 746/1484] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:51:34.745283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a342855-3f99-4987-a9d2-1b2cc1271514 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:34.745846 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a342855-3f99-4987-a9d2-1b2cc1271514 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:34.745846 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a342855-3f99-4987-a9d2-1b2cc1271514 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:34.747183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 739/1485] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:51:34.752502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29ef32-4489-4e00-9cfd-32acb0ab5e62 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:34.752939 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29ef32-4489-4e00-9cfd-32acb0ab5e62 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:34.753042 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29ef32-4489-4e00-9cfd-32acb0ab5e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:34.818178 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-fa29ef32-4489-4e00-9cfd-32acb0ab5e62 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:51:34.819313 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fa29ef32-4489-4e00-9cfd-32acb0ab5e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:34.912655 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 747/1486] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:51:35.205462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a342855-3f99-4987-a9d2-1b2cc1271514 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:51:35.207038 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7a342855-3f99-4987-a9d2-1b2cc1271514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:35.310254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a342855-3f99-4987-a9d2-1b2cc1271514 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:51:35.382532 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:35.384294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60f91d-cbf4-4e1f-af31-b36c719f7c1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:51:35.384888 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60f91d-cbf4-4e1f-af31-b36c719f7c1d service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:35.385163 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60f91d-cbf4-4e1f-af31-b36c719f7c1d service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:35.385429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60f91d-cbf4-4e1f-af31-b36c719f7c1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:35.387011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-fa60f91d-cbf4-4e1f-af31-b36c719f7c1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:35.493834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60f91d-cbf4-4e1f-af31-b36c719f7c1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:51:35.495288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60f91d-cbf4-4e1f-af31-b36c719f7c1d service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:51:35.514101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 740/1487] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:51:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:51:37.540809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd0427-f33b-4a04-b815-a05144bbdd8c None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:37.541314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd0427-f33b-4a04-b815-a05144bbdd8c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:37.541314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd0427-f33b-4a04-b815-a05144bbdd8c None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:37.542628 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 748/1488] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:51:37.547725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f36e0-c110-4f03-8deb-54a0c93c4d7d None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:37.548177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f36e0-c110-4f03-8deb-54a0c93c4d7d None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:37.548292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f36e0-c110-4f03-8deb-54a0c93c4d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:37.613133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-b64f36e0-c110-4f03-8deb-54a0c93c4d7d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:51:37.614354 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b64f36e0-c110-4f03-8deb-54a0c93c4d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:37.666624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:37.709574 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 741/1489] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:51:37.996240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd0427-f33b-4a04-b815-a05144bbdd8c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:51:37.997811 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fbfd0427-f33b-4a04-b815-a05144bbdd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:38.033901 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:38.101756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd0427-f33b-4a04-b815-a05144bbdd8c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:51:38.173693 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc149d3-9388-4076-9dd1-e3d2331a42e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:51:38.174298 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc149d3-9388-4076-9dd1-e3d2331a42e1 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:38.174577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc149d3-9388-4076-9dd1-e3d2331a42e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:38.174849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc149d3-9388-4076-9dd1-e3d2331a42e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:38.177065 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2dc149d3-9388-4076-9dd1-e3d2331a42e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:38.258522 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:38.289656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc149d3-9388-4076-9dd1-e3d2331a42e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:51:38.291170 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc149d3-9388-4076-9dd1-e3d2331a42e1 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:51:38.309462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 749/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:51:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:51:40.608001 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f9753-3424-4161-8c2f-aaa3fe00b54d None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:40.608529 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f9753-3424-4161-8c2f-aaa3fe00b54d None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:40.608529 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f9753-3424-4161-8c2f-aaa3fe00b54d None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:40.609806 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 742/1491] 10.222.0.27 () {54 vars in 927 bytes} [Tue Aug 30 14:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:51:40.615611 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712dbd75-b906-4dd2-9831-6ce1ddff2111 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:40.616050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712dbd75-b906-4dd2-9831-6ce1ddff2111 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:40.616156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712dbd75-b906-4dd2-9831-6ce1ddff2111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:40.681192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-712dbd75-b906-4dd2-9831-6ce1ddff2111 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:51:40.682379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-712dbd75-b906-4dd2-9831-6ce1ddff2111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:40.775218 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 750/1492] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Aug 30 14:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:51:41.064474 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f9753-3424-4161-8c2f-aaa3fe00b54d None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:51:41.066075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-cf9f9753-3424-4161-8c2f-aaa3fe00b54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:41.168108 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f9753-3424-4161-8c2f-aaa3fe00b54d None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:51:41.240329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b30079-0bb2-4aa9-9b9e-3199fd821660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:51:41.240924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b30079-0bb2-4aa9-9b9e-3199fd821660 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:51:41.241202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b30079-0bb2-4aa9-9b9e-3199fd821660 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:51:41.241469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b30079-0bb2-4aa9-9b9e-3199fd821660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:51:41.243045 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d7b30079-0bb2-4aa9-9b9e-3199fd821660 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:51:41.252439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:41.352901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b30079-0bb2-4aa9-9b9e-3199fd821660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:51:41.354252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b30079-0bb2-4aa9-9b9e-3199fd821660 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:51:41.371747 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 743/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:51:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:51:50.773454 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=82 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:51:51.369518 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=50 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:02.067286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e596e52-8227-43e1-aaa5-32acb33293c3 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:02.067904 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e596e52-8227-43e1-aaa5-32acb33293c3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:02.067904 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e596e52-8227-43e1-aaa5-32acb33293c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:02.142153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4e596e52-8227-43e1-aaa5-32acb33293c3 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:02.143424 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4e596e52-8227-43e1-aaa5-32acb33293c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:02.238753 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 751/1494] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:02.247009 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361d63a-561b-405a-852e-932d73e4b039 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:02.247531 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361d63a-561b-405a-852e-932d73e4b039 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:02.247726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361d63a-561b-405a-852e-932d73e4b039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:02.342643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-6361d63a-561b-405a-852e-932d73e4b039 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:02.342643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6361d63a-561b-405a-852e-932d73e4b039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:02.424676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 744/1495] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:02.429341 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e596e52-8227-43e1-aaa5-32acb33293c3 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:02.430551 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4e596e52-8227-43e1-aaa5-32acb33293c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:02.441905 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e596e52-8227-43e1-aaa5-32acb33293c3 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:02.501705 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:02.537432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294eb97-4158-4972-ba08-3fdde5d763ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:02.538277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294eb97-4158-4972-ba08-3fdde5d763ae admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:02.538716 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294eb97-4158-4972-ba08-3fdde5d763ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:02.539314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294eb97-4158-4972-ba08-3fdde5d763ae admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:02.539961 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294eb97-4158-4972-ba08-3fdde5d763ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:53:02.540264 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294eb97-4158-4972-ba08-3fdde5d763ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:02.541277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: WARNING py.warnings [None req-c294eb97-4158-4972-ba08-3fdde5d763ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:53:02.541277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: warnings.warn(msg) Aug 30 14:53:02.541277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: Aug 30 14:53:02.542159 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294eb97-4158-4972-ba08-3fdde5d763ae admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:02.553151 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 752/1496] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:53:02.561550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361d63a-561b-405a-852e-932d73e4b039 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:02.571059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361d63a-561b-405a-852e-932d73e4b039 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:02.631409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:02.665500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c8825-5d85-4d8a-8ec5-9ebc6e018785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:02.666096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c8825-5d85-4d8a-8ec5-9ebc6e018785 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:02.666372 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c8825-5d85-4d8a-8ec5-9ebc6e018785 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:02.666636 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c8825-5d85-4d8a-8ec5-9ebc6e018785 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:02.667333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c8825-5d85-4d8a-8ec5-9ebc6e018785 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:02.668971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: WARNING py.warnings [None req-697c8825-5d85-4d8a-8ec5-9ebc6e018785 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:53:02.668971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: warnings.warn(msg) Aug 30 14:53:02.668971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: Aug 30 14:53:02.670211 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c8825-5d85-4d8a-8ec5-9ebc6e018785 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:02.832792 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 745/1497] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:53:02 2022] POST /identity/v3/projects => generated 377 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:02.837081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294eb97-4158-4972-ba08-3fdde5d763ae admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:02.838593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c294eb97-4158-4972-ba08-3fdde5d763ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:02.872038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294eb97-4158-4972-ba08-3fdde5d763ae admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:02.963879 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f9a63-dfd7-4de0-b946-d0bf9368eddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:02.964401 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f9a63-dfd7-4de0-b946-d0bf9368eddb admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:02.964672 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f9a63-dfd7-4de0-b946-d0bf9368eddb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:02.964935 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f9a63-dfd7-4de0-b946-d0bf9368eddb admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:02.965547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f9a63-dfd7-4de0-b946-d0bf9368eddb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:02.966547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: WARNING py.warnings [None req-bf9f9a63-dfd7-4de0-b946-d0bf9368eddb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:53:02.966547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: warnings.warn(msg) Aug 30 14:53:02.966547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: Aug 30 14:53:02.967787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f9a63-dfd7-4de0-b946-d0bf9368eddb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:03.070129 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 753/1498] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:53:02 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:53:03.074240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c8825-5d85-4d8a-8ec5-9ebc6e018785 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:03.075823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-697c8825-5d85-4d8a-8ec5-9ebc6e018785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:03.137283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c8825-5d85-4d8a-8ec5-9ebc6e018785 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:03.220388 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:03.230994 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ac1ff-ac0f-4d68-bbee-8e1979cde404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:03.231626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ac1ff-ac0f-4d68-bbee-8e1979cde404 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:03.231906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ac1ff-ac0f-4d68-bbee-8e1979cde404 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:03.232173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ac1ff-ac0f-4d68-bbee-8e1979cde404 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:03.232818 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ac1ff-ac0f-4d68-bbee-8e1979cde404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:03.233931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: WARNING py.warnings [None req-d53ac1ff-ac0f-4d68-bbee-8e1979cde404 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:53:03.233931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: warnings.warn(msg) Aug 30 14:53:03.233931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: Aug 30 14:53:03.234824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ac1ff-ac0f-4d68-bbee-8e1979cde404 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:03.248695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 746/1499] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:53:03.253543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f9a63-dfd7-4de0-b946-d0bf9368eddb admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:03.255056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bf9f9a63-dfd7-4de0-b946-d0bf9368eddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:03.266788 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f9a63-dfd7-4de0-b946-d0bf9368eddb admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:03.291434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:03.360765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38410ea2-ded5-44c8-99e7-9fa5cc307210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:03.361368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38410ea2-ded5-44c8-99e7-9fa5cc307210 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:03.361726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38410ea2-ded5-44c8-99e7-9fa5cc307210 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:03.362573 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38410ea2-ded5-44c8-99e7-9fa5cc307210 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:03.363277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38410ea2-ded5-44c8-99e7-9fa5cc307210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:03.364308 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: WARNING py.warnings [None req-38410ea2-ded5-44c8-99e7-9fa5cc307210 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:53:03.364308 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: warnings.warn(msg) Aug 30 14:53:03.364308 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: Aug 30 14:53:03.365224 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38410ea2-ded5-44c8-99e7-9fa5cc307210 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:03.377496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 754/1500] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:53:03.381725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ac1ff-ac0f-4d68-bbee-8e1979cde404 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:03.383249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d53ac1ff-ac0f-4d68-bbee-8e1979cde404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:03.416323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ac1ff-ac0f-4d68-bbee-8e1979cde404 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:03.510779 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52793e-2224-4601-b39e-b1c898e48630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:03.511468 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52793e-2224-4601-b39e-b1c898e48630 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:03.511747 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52793e-2224-4601-b39e-b1c898e48630 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:03.512017 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52793e-2224-4601-b39e-b1c898e48630 admin admin] PATH_INFO: `/v3/projects/074654d755604c0d8168c6adf45567e7/users/6ad7ff2b637341f9b26e451dfcd6e215/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:03.536779 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52793e-2224-4601-b39e-b1c898e48630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=074654d755604c0d8168c6adf45567e7, user_id=6ad7ff2b637341f9b26e451dfcd6e215, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:03.537804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: WARNING py.warnings [None req-ee52793e-2224-4601-b39e-b1c898e48630 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:53:03.537804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: warnings.warn(msg) Aug 30 14:53:03.537804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: Aug 30 14:53:03.539835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52793e-2224-4601-b39e-b1c898e48630 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:03.630387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 747/1501] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:53:03 2022] PUT /identity/v3/projects/074654d755604c0d8168c6adf45567e7/users/6ad7ff2b637341f9b26e451dfcd6e215/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:03.642584 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef5159-3e53-444a-8472-e03e02dd7b47 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:03.642958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef5159-3e53-444a-8472-e03e02dd7b47 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:03.643338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef5159-3e53-444a-8472-e03e02dd7b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:03.701545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e0ef5159-3e53-444a-8472-e03e02dd7b47 None None] MFA Rules not processed for user `6ad7ff2b637341f9b26e451dfcd6e215`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:03.702733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e0ef5159-3e53-444a-8472-e03e02dd7b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:03.781274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 755/1502] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:53:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:03.795341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5109046-4a5b-483d-acbf-3ec99819d1df None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:03.795626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5109046-4a5b-483d-acbf-3ec99819d1df None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:03.795900 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5109046-4a5b-483d-acbf-3ec99819d1df None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:03.797272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 748/1503] 10.222.0.27 () {54 vars in 964 bytes} [Tue Aug 30 14:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:53:03.801933 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dff57b-2c18-458b-907c-b89e127655d4 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:03.802223 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dff57b-2c18-458b-907c-b89e127655d4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:03.802496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dff57b-2c18-458b-907c-b89e127655d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:03.833363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:03.877700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-94dff57b-2c18-458b-907c-b89e127655d4 None None] MFA Rules not processed for user `ef389656cc8546fdb7d9f30329b93dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:03.878900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-94dff57b-2c18-458b-907c-b89e127655d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:03.952695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 756/1504] 10.222.0.27 () {58 vars in 1061 bytes} [Tue Aug 30 14:53:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:03.961051 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5109046-4a5b-483d-acbf-3ec99819d1df None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:03.962561 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d5109046-4a5b-483d-acbf-3ec99819d1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:04.011882 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:04.063957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5109046-4a5b-483d-acbf-3ec99819d1df None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:04.137218 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4118f1f-2a7b-4f5b-a614-5f534cb87857 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:04.137803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4118f1f-2a7b-4f5b-a614-5f534cb87857 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:04.138075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4118f1f-2a7b-4f5b-a614-5f534cb87857 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:04.138342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4118f1f-2a7b-4f5b-a614-5f534cb87857 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:04.139951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c4118f1f-2a7b-4f5b-a614-5f534cb87857 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:04.235747 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4118f1f-2a7b-4f5b-a614-5f534cb87857 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:04.237207 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4118f1f-2a7b-4f5b-a614-5f534cb87857 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:04.240712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:04.257300 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 749/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:53:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:53:04.301767 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bf24c-fed3-422b-bcc2-99c38ecabad1 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:04.302184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bf24c-fed3-422b-bcc2-99c38ecabad1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:04.302321 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bf24c-fed3-422b-bcc2-99c38ecabad1 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:04.303619 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 757/1506] 10.222.0.27 () {52 vars in 830 bytes} [Tue Aug 30 14:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:53:04.307011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4118f1f-2a7b-4f5b-a614-5f534cb87857 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:04.308252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c4118f1f-2a7b-4f5b-a614-5f534cb87857 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:04.408249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4118f1f-2a7b-4f5b-a614-5f534cb87857 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:04.505725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1bd86-5de5-406e-80a5-39d4950df16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:04.506388 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1bd86-5de5-406e-80a5-39d4950df16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:04.506626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1bd86-5de5-406e-80a5-39d4950df16b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:04.506895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1bd86-5de5-406e-80a5-39d4950df16b admin admin] PATH_INFO: `/v3/users/6ad7ff2b637341f9b26e451dfcd6e215` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:04.520312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1bd86-5de5-406e-80a5-39d4950df16b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ad7ff2b637341f9b26e451dfcd6e215)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:04.521479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: WARNING py.warnings [None req-37a1bd86-5de5-406e-80a5-39d4950df16b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:53:04.521479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: warnings.warn(msg) Aug 30 14:53:04.521479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: Aug 30 14:53:04.522730 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1bd86-5de5-406e-80a5-39d4950df16b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:04.532890 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:04.764010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-37a1bd86-5de5-406e-80a5-39d4950df16b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ad7ff2b637341f9b26e451dfcd6e215'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:04.777749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-37a1bd86-5de5-406e-80a5-39d4950df16b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ad7ff2b637341f9b26e451dfcd6e215'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:04.801576 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-37a1bd86-5de5-406e-80a5-39d4950df16b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ad7ff2b637341f9b26e451dfcd6e215'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:04.802030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-37a1bd86-5de5-406e-80a5-39d4950df16b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ad7ff2b637341f9b26e451dfcd6e215'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:04.802547 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-37a1bd86-5de5-406e-80a5-39d4950df16b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ad7ff2b637341f9b26e451dfcd6e215'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:04.869259 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 750/1507] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:53:04 2022] DELETE /identity/v3/users/6ad7ff2b637341f9b26e451dfcd6e215 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:04.873931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6a0e5-a7a6-435f-904b-1239b85d86d1 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:04.874216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6a0e5-a7a6-435f-904b-1239b85d86d1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:04.874508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6a0e5-a7a6-435f-904b-1239b85d86d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:04.930337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-52c6a0e5-a7a6-435f-904b-1239b85d86d1 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:53:04.931558 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-52c6a0e5-a7a6-435f-904b-1239b85d86d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:05.004763 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:05.019342 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 758/1508] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:53:05.026669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1bd86-5de5-406e-80a5-39d4950df16b admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:05.027952 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-37a1bd86-5de5-406e-80a5-39d4950df16b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:05.126296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1bd86-5de5-406e-80a5-39d4950df16b admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:05.181759 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:05.199775 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641af278-d0a5-4c79-b3bf-b343797dcd01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:05.200377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641af278-d0a5-4c79-b3bf-b343797dcd01 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:05.200656 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641af278-d0a5-4c79-b3bf-b343797dcd01 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:05.200931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641af278-d0a5-4c79-b3bf-b343797dcd01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:05.202517 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-641af278-d0a5-4c79-b3bf-b343797dcd01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:05.311471 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641af278-d0a5-4c79-b3bf-b343797dcd01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:05.312941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641af278-d0a5-4c79-b3bf-b343797dcd01 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:05.330676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 751/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:53:05.825426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6a0e5-a7a6-435f-904b-1239b85d86d1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:05.827588 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-52c6a0e5-a7a6-435f-904b-1239b85d86d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:05.838216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6a0e5-a7a6-435f-904b-1239b85d86d1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:05.935388 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8502063-b911-49eb-a110-a8792dd634e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:05.936014 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8502063-b911-49eb-a110-a8792dd634e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:05.936289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8502063-b911-49eb-a110-a8792dd634e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:05.936558 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8502063-b911-49eb-a110-a8792dd634e5 admin admin] PATH_INFO: `/v3/projects/074654d755604c0d8168c6adf45567e7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:05.946312 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8502063-b911-49eb-a110-a8792dd634e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=074654d755604c0d8168c6adf45567e7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:05.947366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: WARNING py.warnings [None req-e8502063-b911-49eb-a110-a8792dd634e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:53:05.947366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: warnings.warn(msg) Aug 30 14:53:05.947366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: Aug 30 14:53:05.948554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8502063-b911-49eb-a110-a8792dd634e5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:06.085509 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e8502063-b911-49eb-a110-a8792dd634e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '074654d755604c0d8168c6adf45567e7', 'request_id': 'req-e8502063-b911-49eb-a110-a8792dd634e5'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:06.099006 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e8502063-b911-49eb-a110-a8792dd634e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '074654d755604c0d8168c6adf45567e7', 'request_id': 'req-e8502063-b911-49eb-a110-a8792dd634e5'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:06.104959 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e8502063-b911-49eb-a110-a8792dd634e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '074654d755604c0d8168c6adf45567e7', 'request_id': 'req-e8502063-b911-49eb-a110-a8792dd634e5'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:06.194356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e8502063-b911-49eb-a110-a8792dd634e5 admin admin] The token cache is being invalidate because project 074654d755604c0d8168c6adf45567e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:53:06.194637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e8502063-b911-49eb-a110-a8792dd634e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:06.195054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e8502063-b911-49eb-a110-a8792dd634e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:53:06.196471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 759/1510] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:53:05 2022] DELETE /identity/v3/projects/074654d755604c0d8168c6adf45567e7 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:53:15.344330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=62 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:16.192304 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:32.605235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef949ee3-6d96-4298-bf7d-777a7cb8bbd3 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:32.605827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef949ee3-6d96-4298-bf7d-777a7cb8bbd3 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:32.605827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef949ee3-6d96-4298-bf7d-777a7cb8bbd3 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:32.607155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:54:32.612837 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5a907-872a-4518-b0bf-9b0694cfcb6f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:32.613251 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5a907-872a-4518-b0bf-9b0694cfcb6f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:32.613399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5a907-872a-4518-b0bf-9b0694cfcb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:32.695145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1fa5a907-872a-4518-b0bf-9b0694cfcb6f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:32.696346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1fa5a907-872a-4518-b0bf-9b0694cfcb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:32.799185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:32.952241 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef949ee3-6d96-4298-bf7d-777a7cb8bbd3 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:32.956582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ef949ee3-6d96-4298-bf7d-777a7cb8bbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:33.070578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef949ee3-6d96-4298-bf7d-777a7cb8bbd3 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:33.076036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab93888-6597-4439-83f2-6b694a178285 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:33.076636 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab93888-6597-4439-83f2-6b694a178285 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:33.076917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab93888-6597-4439-83f2-6b694a178285 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:33.077189 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab93888-6597-4439-83f2-6b694a178285 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:33.085834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab93888-6597-4439-83f2-6b694a178285 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:33.087429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab93888-6597-4439-83f2-6b694a178285 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:33.096570 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 753/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:33.922337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122194b-28d1-47ef-999e-77b1ec7b8d93 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:33.922877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122194b-28d1-47ef-999e-77b1ec7b8d93 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:33.922877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122194b-28d1-47ef-999e-77b1ec7b8d93 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:33.924191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:54:33.930793 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e692985-4954-4276-a1b3-db1876184d7b None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:33.931279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e692985-4954-4276-a1b3-db1876184d7b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:33.931394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e692985-4954-4276-a1b3-db1876184d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:33.951046 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-3e692985-4954-4276-a1b3-db1876184d7b None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:33.952288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-3e692985-4954-4276-a1b3-db1876184d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:33.960164 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:34.114900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122194b-28d1-47ef-999e-77b1ec7b8d93 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:34.129326 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122194b-28d1-47ef-999e-77b1ec7b8d93 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:34.134783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278396c6-4286-41c7-a72e-f5461070a8ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:34.135475 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278396c6-4286-41c7-a72e-f5461070a8ec service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:34.135757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278396c6-4286-41c7-a72e-f5461070a8ec service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:34.136028 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278396c6-4286-41c7-a72e-f5461070a8ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:34.144007 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278396c6-4286-41c7-a72e-f5461070a8ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:34.145586 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278396c6-4286-41c7-a72e-f5461070a8ec service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:34.154858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 762/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:34.744459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6b049-79ae-4de9-8c17-e4f1890e9bfb None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:34.744746 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6b049-79ae-4de9-8c17-e4f1890e9bfb None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:34.745025 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6b049-79ae-4de9-8c17-e4f1890e9bfb None None] PATH_INFO: `/v3` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:34.746303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 755/1517] 10.222.0.59 () {54 vars in 925 bytes} [Tue Aug 30 14:54:34 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:54:34.752580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314633a6-957a-483c-b76c-b39052693648 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:34.752871 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314633a6-957a-483c-b76c-b39052693648 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:34.753628 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314633a6-957a-483c-b76c-b39052693648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:34.811121 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-314633a6-957a-483c-b76c-b39052693648 None None] MFA Rules not processed for user `f4f11655cd51438e9652869f74ffa975`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:34.812338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-314633a6-957a-483c-b76c-b39052693648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:34.829223 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:34.883076 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 763/1518] 10.222.0.59 () {58 vars in 1015 bytes} [Tue Aug 30 14:54:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:38.942571 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5761f7-c76b-4b06-890d-f698e2d3a932 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:38.943210 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5761f7-c76b-4b06-890d-f698e2d3a932 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:38.943210 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5761f7-c76b-4b06-890d-f698e2d3a932 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:38.944357 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 756/1519] 10.222.0.27 () {54 vars in 961 bytes} [Tue Aug 30 14:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:54:38.949579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fe96c-2087-4b73-8d0b-163757410d7b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:38.950032 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fe96c-2087-4b73-8d0b-163757410d7b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:38.950184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fe96c-2087-4b73-8d0b-163757410d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:38.969849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e86fe96c-2087-4b73-8d0b-163757410d7b None None] MFA Rules not processed for user `f4f11655cd51438e9652869f74ffa975`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:38.971060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e86fe96c-2087-4b73-8d0b-163757410d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:38.980057 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 764/1520] 10.222.0.27 () {58 vars in 1058 bytes} [Tue Aug 30 14:54:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:38.987833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5761f7-c76b-4b06-890d-f698e2d3a932 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:39.001378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5761f7-c76b-4b06-890d-f698e2d3a932 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:39.008431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70274482-8da4-4638-9609-449ec4e46b80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4f11655cd51438e9652869f74ffa975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:39.009032 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70274482-8da4-4638-9609-449ec4e46b80 service placement] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:39.009308 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70274482-8da4-4638-9609-449ec4e46b80 service placement] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:39.009577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70274482-8da4-4638-9609-449ec4e46b80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:39.018076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70274482-8da4-4638-9609-449ec4e46b80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:39.020187 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70274482-8da4-4638-9609-449ec4e46b80 service placement] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:39.029605 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 757/1521] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:45.340220 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407beede-84ba-4992-a892-cefb3eb88a26 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:45.340768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407beede-84ba-4992-a892-cefb3eb88a26 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:45.340768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407beede-84ba-4992-a892-cefb3eb88a26 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:45.342145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:54:45.346830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4baae13-8cfe-411b-9c8a-cdef7c65cc3d None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:45.347266 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4baae13-8cfe-411b-9c8a-cdef7c65cc3d None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:45.347379 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4baae13-8cfe-411b-9c8a-cdef7c65cc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:45.366644 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-c4baae13-8cfe-411b-9c8a-cdef7c65cc3d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:45.367859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c4baae13-8cfe-411b-9c8a-cdef7c65cc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:45.375549 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:45.527987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407beede-84ba-4992-a892-cefb3eb88a26 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:45.542260 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407beede-84ba-4992-a892-cefb3eb88a26 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:45.548108 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0235789-4492-479c-bf8a-fcba316620a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:45.548700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0235789-4492-479c-bf8a-fcba316620a4 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:45.548976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0235789-4492-479c-bf8a-fcba316620a4 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:45.549230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0235789-4492-479c-bf8a-fcba316620a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:45.557729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0235789-4492-479c-bf8a-fcba316620a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:45.559408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0235789-4492-479c-bf8a-fcba316620a4 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:45.569395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 766/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:46.384811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff99ad0-6e54-4bc3-9e15-ec07da2cf413 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:46.385361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff99ad0-6e54-4bc3-9e15-ec07da2cf413 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:46.385460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff99ad0-6e54-4bc3-9e15-ec07da2cf413 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:46.386740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:54:46.391662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabca046-8550-4c7b-b54f-85cfd83bf050 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:46.392079 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabca046-8550-4c7b-b54f-85cfd83bf050 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:46.392208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabca046-8550-4c7b-b54f-85cfd83bf050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:46.412163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-fabca046-8550-4c7b-b54f-85cfd83bf050 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:46.413337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fabca046-8550-4c7b-b54f-85cfd83bf050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:46.420939 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:46.573442 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff99ad0-6e54-4bc3-9e15-ec07da2cf413 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:46.588124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff99ad0-6e54-4bc3-9e15-ec07da2cf413 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:46.593427 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101b2d1-c29c-44cc-a0ef-6e8a01fbbd55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:46.594007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101b2d1-c29c-44cc-a0ef-6e8a01fbbd55 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:46.594280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101b2d1-c29c-44cc-a0ef-6e8a01fbbd55 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:46.594547 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101b2d1-c29c-44cc-a0ef-6e8a01fbbd55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:46.602936 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101b2d1-c29c-44cc-a0ef-6e8a01fbbd55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:46.604536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101b2d1-c29c-44cc-a0ef-6e8a01fbbd55 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:46.613582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 760/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:50.390930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91656eab-556d-4866-bf16-5143801ea295 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:50.391532 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91656eab-556d-4866-bf16-5143801ea295 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:50.391532 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91656eab-556d-4866-bf16-5143801ea295 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:50.392838 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:54:50.398001 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92055052-db8c-4036-9fc6-ce54dad3c3dc None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:50.398502 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92055052-db8c-4036-9fc6-ce54dad3c3dc None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:50.398604 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92055052-db8c-4036-9fc6-ce54dad3c3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:50.417871 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-92055052-db8c-4036-9fc6-ce54dad3c3dc None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:50.419041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-92055052-db8c-4036-9fc6-ce54dad3c3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:50.427371 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:50.711441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91656eab-556d-4866-bf16-5143801ea295 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:50.713141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-91656eab-556d-4866-bf16-5143801ea295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:50.790256 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91656eab-556d-4866-bf16-5143801ea295 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:50.793814 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18621c-181e-4108-85ad-25e0d015f193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:50.794405 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18621c-181e-4108-85ad-25e0d015f193 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:50.794679 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18621c-181e-4108-85ad-25e0d015f193 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:50.794944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18621c-181e-4108-85ad-25e0d015f193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:50.804200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18621c-181e-4108-85ad-25e0d015f193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:50.805618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18621c-181e-4108-85ad-25e0d015f193 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:50.814677 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 769/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:52.715013 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97906585-0d18-408a-9f35-7142410fa772 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:52.715576 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97906585-0d18-408a-9f35-7142410fa772 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:52.715576 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97906585-0d18-408a-9f35-7142410fa772 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:52.716848 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:54:52.722677 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e6deb-1fbb-48b3-9e25-a244d1d5c2cd None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:52.723132 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e6deb-1fbb-48b3-9e25-a244d1d5c2cd None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:52.723251 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e6deb-1fbb-48b3-9e25-a244d1d5c2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:52.744394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1a7e6deb-1fbb-48b3-9e25-a244d1d5c2cd None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:52.745586 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1a7e6deb-1fbb-48b3-9e25-a244d1d5c2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:52.753315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:53.045310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97906585-0d18-408a-9f35-7142410fa772 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:53.056734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97906585-0d18-408a-9f35-7142410fa772 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:53.060252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81547440-b305-433b-8820-a5c272b22caa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:53.060839 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81547440-b305-433b-8820-a5c272b22caa service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:53.061118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81547440-b305-433b-8820-a5c272b22caa service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:53.061387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81547440-b305-433b-8820-a5c272b22caa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:53.070072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81547440-b305-433b-8820-a5c272b22caa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:53.071528 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81547440-b305-433b-8820-a5c272b22caa service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:53.081242 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 763/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:56.636864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6ca31-8dfe-49e4-ad90-727bd4130503 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:56.637394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6ca31-8dfe-49e4-ad90-727bd4130503 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:56.637394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6ca31-8dfe-49e4-ad90-727bd4130503 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:56.638663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:54:56.644377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8236d1-ed0f-4a2c-9810-0a700174fa3b None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:56.644813 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8236d1-ed0f-4a2c-9810-0a700174fa3b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:56.644909 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8236d1-ed0f-4a2c-9810-0a700174fa3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:56.652453 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=81 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:56.664563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-bc8236d1-ed0f-4a2c-9810-0a700174fa3b None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:56.665764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-bc8236d1-ed0f-4a2c-9810-0a700174fa3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:56.673506 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:57.226553 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6ca31-8dfe-49e4-ad90-727bd4130503 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:57.228245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d5e6ca31-8dfe-49e4-ad90-727bd4130503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:57.291976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:57.305254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6ca31-8dfe-49e4-ad90-727bd4130503 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:57.308873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fafe378-178d-46f1-91c3-2423e436421d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:57.328979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafe378-178d-46f1-91c3-2423e436421d service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:57.328979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafe378-178d-46f1-91c3-2423e436421d service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:57.328979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fafe378-178d-46f1-91c3-2423e436421d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:57.328979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fafe378-178d-46f1-91c3-2423e436421d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:57.328979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fafe378-178d-46f1-91c3-2423e436421d service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:57.328979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 772/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:59.244514 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc8e7a-f3dd-4648-acc1-015908b11111 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:59.245142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc8e7a-f3dd-4648-acc1-015908b11111 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:59.245142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc8e7a-f3dd-4648-acc1-015908b11111 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:59.246398 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:54:59.251602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1cf56-529c-41c2-b55f-427223438b7a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:59.252013 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1cf56-529c-41c2-b55f-427223438b7a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:59.252148 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1cf56-529c-41c2-b55f-427223438b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:59.271945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-7cf1cf56-529c-41c2-b55f-427223438b7a None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:54:59.273132 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7cf1cf56-529c-41c2-b55f-427223438b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:59.281224 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:54:59.827140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc8e7a-f3dd-4648-acc1-015908b11111 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:59.828814 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-75dc8e7a-f3dd-4648-acc1-015908b11111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:59.841637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc8e7a-f3dd-4648-acc1-015908b11111 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:59.845136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc287a-6abe-435d-a72d-caf6838363cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:59.845717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc287a-6abe-435d-a72d-caf6838363cf service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:59.845993 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc287a-6abe-435d-a72d-caf6838363cf service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:59.846323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc287a-6abe-435d-a72d-caf6838363cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:59.854958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc287a-6abe-435d-a72d-caf6838363cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:59.856392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc287a-6abe-435d-a72d-caf6838363cf service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:59.866404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 766/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:55:09.266687 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=11 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:09.861626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=8 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:23.667228 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f2d58-ef71-4706-b477-86bbc8ab46a8 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:23.667895 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f2d58-ef71-4706-b477-86bbc8ab46a8 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:23.667895 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f2d58-ef71-4706-b477-86bbc8ab46a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:23.696765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-aa0f2d58-ef71-4706-b477-86bbc8ab46a8 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:23.698240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-aa0f2d58-ef71-4706-b477-86bbc8ab46a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:23.712583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 774/1540] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:23.722433 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7777e-1555-4eff-9d68-ecd2e9e8c415 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:23.724242 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7777e-1555-4eff-9d68-ecd2e9e8c415 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:23.724242 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7777e-1555-4eff-9d68-ecd2e9e8c415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:23.752429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-33b7777e-1555-4eff-9d68-ecd2e9e8c415 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:23.753705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-33b7777e-1555-4eff-9d68-ecd2e9e8c415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:23.765327 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 767/1541] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:23.771138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f2d58-ef71-4706-b477-86bbc8ab46a8 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:23.782062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f2d58-ef71-4706-b477-86bbc8ab46a8 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:23.791190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b60ad-f855-462f-aa22-70e006930173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:23.791822 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b60ad-f855-462f-aa22-70e006930173 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:23.792105 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b60ad-f855-462f-aa22-70e006930173 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:23.792380 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b60ad-f855-462f-aa22-70e006930173 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:23.793047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b60ad-f855-462f-aa22-70e006930173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:23.793353 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b60ad-f855-462f-aa22-70e006930173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:23.795030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b60ad-f855-462f-aa22-70e006930173 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:23.810004 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 775/1542] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:23.818397 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7777e-1555-4eff-9d68-ecd2e9e8c415 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:23.829109 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7777e-1555-4eff-9d68-ecd2e9e8c415 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:23.837175 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaf2d8-e02d-4225-818f-a441b21b6efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:23.837936 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaf2d8-e02d-4225-818f-a441b21b6efe admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:23.838221 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaf2d8-e02d-4225-818f-a441b21b6efe admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:23.838494 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaf2d8-e02d-4225-818f-a441b21b6efe admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:23.839184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eaf2d8-e02d-4225-818f-a441b21b6efe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:23.841154 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eaf2d8-e02d-4225-818f-a441b21b6efe admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:23.903653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7dbbe-bf0b-4a5c-a644-5a0e619ed33a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:23.903653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7dbbe-bf0b-4a5c-a644-5a0e619ed33a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:23.903653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7dbbe-bf0b-4a5c-a644-5a0e619ed33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:23.929799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a5c7dbbe-bf0b-4a5c-a644-5a0e619ed33a None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:23.931255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a5c7dbbe-bf0b-4a5c-a644-5a0e619ed33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:23.942591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 776/1543] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:56:23.952101 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699c377-5da3-4414-aa06-06ca004f299f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:23.952399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699c377-5da3-4414-aa06-06ca004f299f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:23.952680 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699c377-5da3-4414-aa06-06ca004f299f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:23.979140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-7699c377-5da3-4414-aa06-06ca004f299f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:23.980495 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7699c377-5da3-4414-aa06-06ca004f299f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:23.991175 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 777/1544] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:56:23.996022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699c377-5da3-4414-aa06-06ca004f299f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.011190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699c377-5da3-4414-aa06-06ca004f299f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.019503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce18225-45b2-49bf-a446-e432dbfa6bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.019503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce18225-45b2-49bf-a446-e432dbfa6bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.019503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce18225-45b2-49bf-a446-e432dbfa6bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.019823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 768/1545] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:23 2022] POST /identity/v3/projects => generated 377 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:24.019951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce18225-45b2-49bf-a446-e432dbfa6bb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.019951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce18225-45b2-49bf-a446-e432dbfa6bb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:24.019951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce18225-45b2-49bf-a446-e432dbfa6bb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.021501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce18225-45b2-49bf-a446-e432dbfa6bb1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.031148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaf2d8-e02d-4225-818f-a441b21b6efe admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.040277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 778/1546] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:24.042021 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88028c-34fd-424e-94da-0c12a18ee22b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.042307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88028c-34fd-424e-94da-0c12a18ee22b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.042583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88028c-34fd-424e-94da-0c12a18ee22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.070873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0d88028c-34fd-424e-94da-0c12a18ee22b None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:24.072582 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0d88028c-34fd-424e-94da-0c12a18ee22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:24.148227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaf2d8-e02d-4225-818f-a441b21b6efe admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.153376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa142a9-f043-4960-a67f-b0d6ee9a57a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.154042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa142a9-f043-4960-a67f-b0d6ee9a57a7 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.154252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa142a9-f043-4960-a67f-b0d6ee9a57a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.154518 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa142a9-f043-4960-a67f-b0d6ee9a57a7 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.155343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa142a9-f043-4960-a67f-b0d6ee9a57a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.156350 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: WARNING py.warnings [None req-0aa142a9-f043-4960-a67f-b0d6ee9a57a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:56:24.156350 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: warnings.warn(msg) Aug 30 14:56:24.156350 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: Aug 30 14:56:24.157553 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa142a9-f043-4960-a67f-b0d6ee9a57a7 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.158466 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 779/1547] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:24.159149 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88028c-34fd-424e-94da-0c12a18ee22b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.169374 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88028c-34fd-424e-94da-0c12a18ee22b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.174765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f29f5-a065-4165-add2-b24a04d9537f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.175399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f29f5-a065-4165-add2-b24a04d9537f admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.175681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f29f5-a065-4165-add2-b24a04d9537f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.175951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f29f5-a065-4165-add2-b24a04d9537f admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.176622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f29f5-a065-4165-add2-b24a04d9537f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.177647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: WARNING py.warnings [None req-df3f29f5-a065-4165-add2-b24a04d9537f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:56:24.177647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: warnings.warn(msg) Aug 30 14:56:24.177647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: Aug 30 14:56:24.178851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f29f5-a065-4165-add2-b24a04d9537f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.285075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 769/1548] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/users => generated 289 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:56:24.286356 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bd499-d4f1-4c1b-b19c-20435e34951c None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.286636 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bd499-d4f1-4c1b-b19c-20435e34951c None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.286902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bd499-d4f1-4c1b-b19c-20435e34951c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.305432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-b91bd499-d4f1-4c1b-b19c-20435e34951c None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:24.306649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b91bd499-d4f1-4c1b-b19c-20435e34951c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:24.314790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 770/1549] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:24.314944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 780/1550] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/projects => generated 389 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:24.316179 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cbcfb-f911-4c47-9c86-c19a40f7c943 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.316470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cbcfb-f911-4c47-9c86-c19a40f7c943 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.316584 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3636a6-8d9b-4c04-ae8e-a260aa7b5b92 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.316751 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cbcfb-f911-4c47-9c86-c19a40f7c943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.317357 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3636a6-8d9b-4c04-ae8e-a260aa7b5b92 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.317577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3636a6-8d9b-4c04-ae8e-a260aa7b5b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.337358 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-7f3636a6-8d9b-4c04-ae8e-a260aa7b5b92 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:24.343587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7f3636a6-8d9b-4c04-ae8e-a260aa7b5b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:24.343727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-1d9cbcfb-f911-4c47-9c86-c19a40f7c943 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:24.343727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1d9cbcfb-f911-4c47-9c86-c19a40f7c943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:24.397482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 781/1551] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:24.398562 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4680d02-c97a-4fc6-b52e-5fe21d7dadf2 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.398841 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4680d02-c97a-4fc6-b52e-5fe21d7dadf2 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.399127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4680d02-c97a-4fc6-b52e-5fe21d7dadf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.400377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 771/1552] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:24.401486 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090f0a4-eb0a-46b3-a29b-12d95f73ccdf None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.401879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090f0a4-eb0a-46b3-a29b-12d95f73ccdf None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.402154 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090f0a4-eb0a-46b3-a29b-12d95f73ccdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.423753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-c090f0a4-eb0a-46b3-a29b-12d95f73ccdf None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:24.423753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c090f0a4-eb0a-46b3-a29b-12d95f73ccdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:24.424241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e4680d02-c97a-4fc6-b52e-5fe21d7dadf2 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:24.425524 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e4680d02-c97a-4fc6-b52e-5fe21d7dadf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:24.431292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 772/1553] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:24.434296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 782/1554] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:24.434925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4680d02-c97a-4fc6-b52e-5fe21d7dadf2 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.436500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d1846-e29d-48cc-8ca8-17b3bd491533 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.436788 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d1846-e29d-48cc-8ca8-17b3bd491533 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.437061 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d1846-e29d-48cc-8ca8-17b3bd491533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.451030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4680d02-c97a-4fc6-b52e-5fe21d7dadf2 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.455737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45654dce-8d99-4ffb-a623-a820c749d5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.456334 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45654dce-8d99-4ffb-a623-a820c749d5ce admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.456614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45654dce-8d99-4ffb-a623-a820c749d5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.456882 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45654dce-8d99-4ffb-a623-a820c749d5ce admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.457413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-8b2d1846-e29d-48cc-8ca8-17b3bd491533 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:24.457689 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45654dce-8d99-4ffb-a623-a820c749d5ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.458599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8b2d1846-e29d-48cc-8ca8-17b3bd491533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:24.459458 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45654dce-8d99-4ffb-a623-a820c749d5ce admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.466444 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 773/1555] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:24.467778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593cc28-496b-47d4-998e-bb87d9c0b596 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.468165 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593cc28-496b-47d4-998e-bb87d9c0b596 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.468440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593cc28-496b-47d4-998e-bb87d9c0b596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.471522 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 783/1556] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:24.471522 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45654dce-8d99-4ffb-a623-a820c749d5ce admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.481445 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45654dce-8d99-4ffb-a623-a820c749d5ce admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.486518 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87117ee9-ee38-47b2-bb95-6a30e2038776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.487128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87117ee9-ee38-47b2-bb95-6a30e2038776 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.487415 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87117ee9-ee38-47b2-bb95-6a30e2038776 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.487686 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87117ee9-ee38-47b2-bb95-6a30e2038776 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.488316 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87117ee9-ee38-47b2-bb95-6a30e2038776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.490861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87117ee9-ee38-47b2-bb95-6a30e2038776 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.491698 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-9593cc28-496b-47d4-998e-bb87d9c0b596 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:24.492887 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9593cc28-496b-47d4-998e-bb87d9c0b596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:24.500821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 774/1557] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:24.502131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc22c4f-03fb-4f13-9217-6b6a33151938 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.502730 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc22c4f-03fb-4f13-9217-6b6a33151938 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.503067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc22c4f-03fb-4f13-9217-6b6a33151938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.522868 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-ddc22c4f-03fb-4f13-9217-6b6a33151938 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:24.524162 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ddc22c4f-03fb-4f13-9217-6b6a33151938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:24.532477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 775/1558] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:24.533977 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38186f8e-d085-4f49-892a-8bf14ae58556 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.534258 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38186f8e-d085-4f49-892a-8bf14ae58556 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.534527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38186f8e-d085-4f49-892a-8bf14ae58556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.554967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-38186f8e-d085-4f49-892a-8bf14ae58556 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:24.556224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-38186f8e-d085-4f49-892a-8bf14ae58556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:24.564769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 776/1559] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:24.565531 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38186f8e-d085-4f49-892a-8bf14ae58556 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.575244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38186f8e-d085-4f49-892a-8bf14ae58556 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.579690 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdce6d-1985-496f-ada4-4ccf07bd8465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.580249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdce6d-1985-496f-ada4-4ccf07bd8465 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.580523 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdce6d-1985-496f-ada4-4ccf07bd8465 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.580788 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdce6d-1985-496f-ada4-4ccf07bd8465 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.581416 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdce6d-1985-496f-ada4-4ccf07bd8465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:24.581711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdce6d-1985-496f-ada4-4ccf07bd8465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.582702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: WARNING py.warnings [None req-a4cdce6d-1985-496f-ada4-4ccf07bd8465 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:56:24.582702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: warnings.warn(msg) Aug 30 14:56:24.582702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: Aug 30 14:56:24.583599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdce6d-1985-496f-ada4-4ccf07bd8465 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.591921 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:24.595028 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 777/1560] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:24.598296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314cc12-992e-40cf-ac51-029d468c20dc None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.598296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314cc12-992e-40cf-ac51-029d468c20dc None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.598296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314cc12-992e-40cf-ac51-029d468c20dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.615722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-4314cc12-992e-40cf-ac51-029d468c20dc None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:24.616891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-4314cc12-992e-40cf-ac51-029d468c20dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:24.625050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 778/1561] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:24.626184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc88b3-eec8-4dd1-bee8-85b3de674ac9 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.626470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc88b3-eec8-4dd1-bee8-85b3de674ac9 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.626744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc88b3-eec8-4dd1-bee8-85b3de674ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.647681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-14dc88b3-eec8-4dd1-bee8-85b3de674ac9 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:24.648847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-14dc88b3-eec8-4dd1-bee8-85b3de674ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:24.656534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 779/1562] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:24.657275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc88b3-eec8-4dd1-bee8-85b3de674ac9 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.666231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc88b3-eec8-4dd1-bee8-85b3de674ac9 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.670707 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ca1db-146f-47ce-96d4-edfe6a40944e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.675359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ca1db-146f-47ce-96d4-edfe6a40944e admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.675359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ca1db-146f-47ce-96d4-edfe6a40944e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.675359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ca1db-146f-47ce-96d4-edfe6a40944e admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.675359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ca1db-146f-47ce-96d4-edfe6a40944e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.675359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ca1db-146f-47ce-96d4-edfe6a40944e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.684067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 780/1563] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:24.684829 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ca1db-146f-47ce-96d4-edfe6a40944e admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.694320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ca1db-146f-47ce-96d4-edfe6a40944e admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.698815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899a3bf-d0d2-4bfe-88e6-2f6e3e25b763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.699386 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899a3bf-d0d2-4bfe-88e6-2f6e3e25b763 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.699662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899a3bf-d0d2-4bfe-88e6-2f6e3e25b763 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.699929 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899a3bf-d0d2-4bfe-88e6-2f6e3e25b763 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.700539 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899a3bf-d0d2-4bfe-88e6-2f6e3e25b763 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:24.700833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899a3bf-d0d2-4bfe-88e6-2f6e3e25b763 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.702344 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899a3bf-d0d2-4bfe-88e6-2f6e3e25b763 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.715032 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 781/1564] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:24.715817 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899a3bf-d0d2-4bfe-88e6-2f6e3e25b763 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.724817 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899a3bf-d0d2-4bfe-88e6-2f6e3e25b763 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.733136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e594e-1901-4e07-94be-ec118a48e33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.733136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e594e-1901-4e07-94be-ec118a48e33d admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.733136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e594e-1901-4e07-94be-ec118a48e33d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.733136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e594e-1901-4e07-94be-ec118a48e33d admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.733520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e594e-1901-4e07-94be-ec118a48e33d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:24.733520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e594e-1901-4e07-94be-ec118a48e33d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.733520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e594e-1901-4e07-94be-ec118a48e33d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.744494 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 782/1565] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:24.745305 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e594e-1901-4e07-94be-ec118a48e33d admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.754215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e594e-1901-4e07-94be-ec118a48e33d admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.759247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86585ea-1f94-42c4-aea2-28e86fc20d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.759785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86585ea-1f94-42c4-aea2-28e86fc20d67 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.760057 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86585ea-1f94-42c4-aea2-28e86fc20d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.760321 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86585ea-1f94-42c4-aea2-28e86fc20d67 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.760922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86585ea-1f94-42c4-aea2-28e86fc20d67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:24.761211 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86585ea-1f94-42c4-aea2-28e86fc20d67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.762726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86585ea-1f94-42c4-aea2-28e86fc20d67 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.774117 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 783/1566] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:24.774666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86585ea-1f94-42c4-aea2-28e86fc20d67 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.783900 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86585ea-1f94-42c4-aea2-28e86fc20d67 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.788901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f740e44-566f-4780-876a-c14f65df6050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.789449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f740e44-566f-4780-876a-c14f65df6050 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.789706 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f740e44-566f-4780-876a-c14f65df6050 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.789982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f740e44-566f-4780-876a-c14f65df6050 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.790577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f740e44-566f-4780-876a-c14f65df6050 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.792230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:24.793459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f740e44-566f-4780-876a-c14f65df6050 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.799127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 784/1567] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/users => generated 295 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:24.799931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87117ee9-ee38-47b2-bb95-6a30e2038776 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.811932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87117ee9-ee38-47b2-bb95-6a30e2038776 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.816479 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53924f-2733-473a-932d-cdc469bf1b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.817066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53924f-2733-473a-932d-cdc469bf1b90 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.817344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53924f-2733-473a-932d-cdc469bf1b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.817614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53924f-2733-473a-932d-cdc469bf1b90 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.818259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53924f-2733-473a-932d-cdc469bf1b90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:24.818556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53924f-2733-473a-932d-cdc469bf1b90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.820305 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53924f-2733-473a-932d-cdc469bf1b90 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.833245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 785/1568] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:24.834020 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53924f-2733-473a-932d-cdc469bf1b90 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.850406 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53924f-2733-473a-932d-cdc469bf1b90 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.855214 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6b44d-a6ac-4fb0-acba-ea3ac75d9f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.855808 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6b44d-a6ac-4fb0-acba-ea3ac75d9f74 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.856084 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6b44d-a6ac-4fb0-acba-ea3ac75d9f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.856354 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6b44d-a6ac-4fb0-acba-ea3ac75d9f74 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.856991 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6b44d-a6ac-4fb0-acba-ea3ac75d9f74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:24.857291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6b44d-a6ac-4fb0-acba-ea3ac75d9f74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.858880 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6b44d-a6ac-4fb0-acba-ea3ac75d9f74 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.871832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 786/1569] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:24.872627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6b44d-a6ac-4fb0-acba-ea3ac75d9f74 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.881068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 784/1570] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:24.881888 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f740e44-566f-4780-876a-c14f65df6050 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:24.886799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6b44d-a6ac-4fb0-acba-ea3ac75d9f74 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.942507 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f740e44-566f-4780-876a-c14f65df6050 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:24.943594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039df8ca-8cb0-40c0-95d9-77d800529e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.944250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039df8ca-8cb0-40c0-95d9-77d800529e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.944522 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039df8ca-8cb0-40c0-95d9-77d800529e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.944797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039df8ca-8cb0-40c0-95d9-77d800529e0a admin admin] PATH_INFO: `/v3/projects/4261589fa8ec4b4bb680a7b3636bbcd1/users/07c1825673d7451ba7eb551fc1983228/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.947517 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e16c64-fe66-443a-a72f-3eb3b0c0c3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:24.948143 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e16c64-fe66-443a-a72f-3eb3b0c0c3cf admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:24.948417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e16c64-fe66-443a-a72f-3eb3b0c0c3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:24.948682 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e16c64-fe66-443a-a72f-3eb3b0c0c3cf admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:24.949314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e16c64-fe66-443a-a72f-3eb3b0c0c3cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.951474 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e16c64-fe66-443a-a72f-3eb3b0c0c3cf admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:24.958780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039df8ca-8cb0-40c0-95d9-77d800529e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4261589fa8ec4b4bb680a7b3636bbcd1, user_id=07c1825673d7451ba7eb551fc1983228, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:24.959835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: WARNING py.warnings [None req-039df8ca-8cb0-40c0-95d9-77d800529e0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:56:24.959835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: warnings.warn(msg) Aug 30 14:56:24.959835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: Aug 30 14:56:24.961764 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039df8ca-8cb0-40c0-95d9-77d800529e0a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:25.026852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 787/1571] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:24 2022] PUT /identity/v3/projects/4261589fa8ec4b4bb680a7b3636bbcd1/users/07c1825673d7451ba7eb551fc1983228/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:25.027674 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039df8ca-8cb0-40c0-95d9-77d800529e0a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:25.061042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 785/1572] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:24 2022] POST /identity/v3/projects => generated 403 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:25.061042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e16c64-fe66-443a-a72f-3eb3b0c0c3cf admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:25.090805 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039df8ca-8cb0-40c0-95d9-77d800529e0a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:25.122307 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e16c64-fe66-443a-a72f-3eb3b0c0c3cf admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:25.127003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12b0a7-060c-4ccb-8097-bdf442d109f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:25.128835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12b0a7-060c-4ccb-8097-bdf442d109f3 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:25.128835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12b0a7-060c-4ccb-8097-bdf442d109f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:25.128835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12b0a7-060c-4ccb-8097-bdf442d109f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:25.128835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12b0a7-060c-4ccb-8097-bdf442d109f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:25.130789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12b0a7-060c-4ccb-8097-bdf442d109f3 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:25.135346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:25.152058 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce5ccb-ace0-4e64-93ff-a5e429dc3d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:25.153102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce5ccb-ace0-4e64-93ff-a5e429dc3d1e admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:25.153592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce5ccb-ace0-4e64-93ff-a5e429dc3d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:25.154083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce5ccb-ace0-4e64-93ff-a5e429dc3d1e admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:25.155255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce5ccb-ace0-4e64-93ff-a5e429dc3d1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:25.158443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce5ccb-ace0-4e64-93ff-a5e429dc3d1e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:25.164882 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:25.233150 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 786/1573] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:25 2022] POST /identity/v3/projects => generated 389 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:25.236749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12b0a7-060c-4ccb-8097-bdf442d109f3 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:25.275168 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 788/1574] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:25 2022] POST /identity/v3/projects => generated 355 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:25.276187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce5ccb-ace0-4e64-93ff-a5e429dc3d1e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:25.297175 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12b0a7-060c-4ccb-8097-bdf442d109f3 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:25.338261 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce5ccb-ace0-4e64-93ff-a5e429dc3d1e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:25.342932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff10c90-733e-40a8-8ef0-ba861756d246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:25.343566 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff10c90-733e-40a8-8ef0-ba861756d246 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:25.343849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff10c90-733e-40a8-8ef0-ba861756d246 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:25.344119 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff10c90-733e-40a8-8ef0-ba861756d246 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:25.344755 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff10c90-733e-40a8-8ef0-ba861756d246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:25.346793 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff10c90-733e-40a8-8ef0-ba861756d246 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:25.351412 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360b8dc-a46b-460a-995b-a7a84857d62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:25.352018 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360b8dc-a46b-460a-995b-a7a84857d62f admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:25.352296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360b8dc-a46b-460a-995b-a7a84857d62f admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:25.352566 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360b8dc-a46b-460a-995b-a7a84857d62f admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:25.353208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360b8dc-a46b-460a-995b-a7a84857d62f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:25.354910 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360b8dc-a46b-460a-995b-a7a84857d62f admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:25.365572 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 787/1575] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:25.368584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360b8dc-a46b-460a-995b-a7a84857d62f admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:25.375956 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360b8dc-a46b-460a-995b-a7a84857d62f admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:25.380461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838b037-6f9b-4f74-928c-74a942e72a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:25.381020 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838b037-6f9b-4f74-928c-74a942e72a2b admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:25.381314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838b037-6f9b-4f74-928c-74a942e72a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:25.381582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838b037-6f9b-4f74-928c-74a942e72a2b admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:25.382189 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838b037-6f9b-4f74-928c-74a942e72a2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:25.384240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838b037-6f9b-4f74-928c-74a942e72a2b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:25.466745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 789/1576] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:25 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:25.466745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff10c90-733e-40a8-8ef0-ba861756d246 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:25.497153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 788/1577] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:25 2022] POST /identity/v3/projects => generated 393 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:25.497153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fed565-11cb-4853-bd1b-d1b224e12aec None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:25.497153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fed565-11cb-4853-bd1b-d1b224e12aec None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:25.497153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fed565-11cb-4853-bd1b-d1b224e12aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:25.515197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-b7fed565-11cb-4853-bd1b-d1b224e12aec None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:25.516405 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b7fed565-11cb-4853-bd1b-d1b224e12aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:25.530917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff10c90-733e-40a8-8ef0-ba861756d246 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:25.575828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 789/1578] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:25.579066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fed565-11cb-4853-bd1b-d1b224e12aec None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:25.587023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fed565-11cb-4853-bd1b-d1b224e12aec None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:25.589577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd38e81-368d-45f8-88b7-26bd3d9ffa29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:25.590212 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd38e81-368d-45f8-88b7-26bd3d9ffa29 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:25.590490 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd38e81-368d-45f8-88b7-26bd3d9ffa29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:25.590758 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd38e81-368d-45f8-88b7-26bd3d9ffa29 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:25.591427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd38e81-368d-45f8-88b7-26bd3d9ffa29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:25.591851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b79127-1048-4501-9469-31fc18c4a5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:25.592496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b79127-1048-4501-9469-31fc18c4a5c2 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:25.592768 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b79127-1048-4501-9469-31fc18c4a5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:25.593033 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b79127-1048-4501-9469-31fc18c4a5c2 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:25.593356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd38e81-368d-45f8-88b7-26bd3d9ffa29 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:25.593651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b79127-1048-4501-9469-31fc18c4a5c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:25.595597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b79127-1048-4501-9469-31fc18c4a5c2 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:25.603824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:25.774507 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 790/1580] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:25 2022] POST /identity/v3/users => generated 302 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:25.775158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 790/1580] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:25 2022] POST /identity/v3/users => generated 289 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:25.775330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd38e81-368d-45f8-88b7-26bd3d9ffa29 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:25.775482 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b79127-1048-4501-9469-31fc18c4a5c2 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:25.785500 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd38e81-368d-45f8-88b7-26bd3d9ffa29 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:25.785818 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b79127-1048-4501-9469-31fc18c4a5c2 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:25.790954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc75a1d-2bbb-4095-82e8-4babc43b5d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:25.794671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a22a54-eedf-4481-bcf7-e69eccd484f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:25.795137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc75a1d-2bbb-4095-82e8-4babc43b5d53 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:25.795276 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a22a54-eedf-4481-bcf7-e69eccd484f1 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:25.795379 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc75a1d-2bbb-4095-82e8-4babc43b5d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:25.795480 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a22a54-eedf-4481-bcf7-e69eccd484f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:25.795568 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc75a1d-2bbb-4095-82e8-4babc43b5d53 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:25.795651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a22a54-eedf-4481-bcf7-e69eccd484f1 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:25.795744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc75a1d-2bbb-4095-82e8-4babc43b5d53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:25.795829 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a22a54-eedf-4481-bcf7-e69eccd484f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:25.795919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc75a1d-2bbb-4095-82e8-4babc43b5d53 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:25.796004 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a22a54-eedf-4481-bcf7-e69eccd484f1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:25.804269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:25.898969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 791/1581] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:25 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:25.898969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc75a1d-2bbb-4095-82e8-4babc43b5d53 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:25.899633 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 791/1582] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:25 2022] POST /identity/v3/users => generated 278 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:25.900279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a22a54-eedf-4481-bcf7-e69eccd484f1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:25.906489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc75a1d-2bbb-4095-82e8-4babc43b5d53 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:25.910008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a22a54-eedf-4481-bcf7-e69eccd484f1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:25.910995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083b30d-8ec5-4a50-9190-91e420c7307e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:25.911638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083b30d-8ec5-4a50-9190-91e420c7307e admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:25.911911 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083b30d-8ec5-4a50-9190-91e420c7307e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:25.912178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083b30d-8ec5-4a50-9190-91e420c7307e admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:25.912808 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083b30d-8ec5-4a50-9190-91e420c7307e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:25.914437 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083b30d-8ec5-4a50-9190-91e420c7307e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:25.914560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1e4c7-12a4-435c-b54f-574bf369e310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:25.915229 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1e4c7-12a4-435c-b54f-574bf369e310 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:25.915648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1e4c7-12a4-435c-b54f-574bf369e310 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:25.915797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1e4c7-12a4-435c-b54f-574bf369e310 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:25.916381 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1e4c7-12a4-435c-b54f-574bf369e310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:25.918262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1e4c7-12a4-435c-b54f-574bf369e310 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:25.924972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 792/1583] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:25.926625 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083b30d-8ec5-4a50-9190-91e420c7307e admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:25.935833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083b30d-8ec5-4a50-9190-91e420c7307e admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:25.940441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d85e79-3512-464b-89aa-ae24dfa1ef1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:25.941013 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d85e79-3512-464b-89aa-ae24dfa1ef1d admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:25.941293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d85e79-3512-464b-89aa-ae24dfa1ef1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:25.941564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d85e79-3512-464b-89aa-ae24dfa1ef1d admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:25.942179 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d85e79-3512-464b-89aa-ae24dfa1ef1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:25.944765 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d85e79-3512-464b-89aa-ae24dfa1ef1d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:25.996830 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 792/1584] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:25 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:25.997666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1e4c7-12a4-435c-b54f-574bf369e310 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:25.999292 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c1a1e4c7-12a4-435c-b54f-574bf369e310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:26.016516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 793/1585] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:25 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:26.020720 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d85e79-3512-464b-89aa-ae24dfa1ef1d admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.030148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d85e79-3512-464b-89aa-ae24dfa1ef1d admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.035076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9ef12-69f2-4401-a218-fd1545e7b9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.035795 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9ef12-69f2-4401-a218-fd1545e7b9f6 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.036194 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9ef12-69f2-4401-a218-fd1545e7b9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.036558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9ef12-69f2-4401-a218-fd1545e7b9f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.037458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9ef12-69f2-4401-a218-fd1545e7b9f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.039208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9ef12-69f2-4401-a218-fd1545e7b9f6 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.049255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 794/1586] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:26.050297 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9ef12-69f2-4401-a218-fd1545e7b9f6 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.057713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1e4c7-12a4-435c-b54f-574bf369e310 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.060230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9ef12-69f2-4401-a218-fd1545e7b9f6 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.061366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a3e2d-6ddc-4bda-9a24-2233f619b561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.061996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a3e2d-6ddc-4bda-9a24-2233f619b561 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.062271 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a3e2d-6ddc-4bda-9a24-2233f619b561 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.062540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a3e2d-6ddc-4bda-9a24-2233f619b561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.064865 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec00b78-4fc5-4128-b517-8a752416e2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.065426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec00b78-4fc5-4128-b517-8a752416e2bc admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.065787 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec00b78-4fc5-4128-b517-8a752416e2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.066060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec00b78-4fc5-4128-b517-8a752416e2bc admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.066682 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec00b78-4fc5-4128-b517-8a752416e2bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.068311 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec00b78-4fc5-4128-b517-8a752416e2bc admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.072028 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a3e2d-6ddc-4bda-9a24-2233f619b561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.073458 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09a3e2d-6ddc-4bda-9a24-2233f619b561 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.080332 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 795/1587] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:26.081153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec00b78-4fc5-4128-b517-8a752416e2bc admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.082799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 793/1588] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:26.083626 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a3e2d-6ddc-4bda-9a24-2233f619b561 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.093258 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec00b78-4fc5-4128-b517-8a752416e2bc admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.093770 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a3e2d-6ddc-4bda-9a24-2233f619b561 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.098555 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336aeef-6634-4d43-a900-f52e79483270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.099160 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3180afe-0dfe-4aaa-87ed-da56d5e35c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.099300 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336aeef-6634-4d43-a900-f52e79483270 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.099441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336aeef-6634-4d43-a900-f52e79483270 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.099989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336aeef-6634-4d43-a900-f52e79483270 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.100101 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3180afe-0dfe-4aaa-87ed-da56d5e35c18 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.100479 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3180afe-0dfe-4aaa-87ed-da56d5e35c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.100846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3180afe-0dfe-4aaa-87ed-da56d5e35c18 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.101366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336aeef-6634-4d43-a900-f52e79483270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.101677 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3180afe-0dfe-4aaa-87ed-da56d5e35c18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.103068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336aeef-6634-4d43-a900-f52e79483270 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.103771 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3180afe-0dfe-4aaa-87ed-da56d5e35c18 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.114629 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 794/1589] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:26.118092 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 796/1590] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:26.118202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3180afe-0dfe-4aaa-87ed-da56d5e35c18 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.118282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336aeef-6634-4d43-a900-f52e79483270 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.126067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3180afe-0dfe-4aaa-87ed-da56d5e35c18 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.126570 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336aeef-6634-4d43-a900-f52e79483270 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.130902 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ee6b1-679e-4941-8cc2-826d2f434050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.131048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adef1d3-f148-4fe4-a203-21cc6eec9114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.132127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ee6b1-679e-4941-8cc2-826d2f434050 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.132264 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adef1d3-f148-4fe4-a203-21cc6eec9114 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.132372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ee6b1-679e-4941-8cc2-826d2f434050 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.132485 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adef1d3-f148-4fe4-a203-21cc6eec9114 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.132642 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ee6b1-679e-4941-8cc2-826d2f434050 admin admin] PATH_INFO: `/v3/projects/af84072cdab4455a95655c3c5628e3ac/users/7dc5cd44b1b34a7ebbdca2c062715431/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.132770 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adef1d3-f148-4fe4-a203-21cc6eec9114 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.133319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adef1d3-f148-4fe4-a203-21cc6eec9114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.134902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adef1d3-f148-4fe4-a203-21cc6eec9114 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.144727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 797/1591] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:26.145514 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adef1d3-f148-4fe4-a203-21cc6eec9114 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.150967 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ee6b1-679e-4941-8cc2-826d2f434050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af84072cdab4455a95655c3c5628e3ac, user_id=7dc5cd44b1b34a7ebbdca2c062715431, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.154250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ee6b1-679e-4941-8cc2-826d2f434050 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.154796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adef1d3-f148-4fe4-a203-21cc6eec9114 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.159931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd25e2c-c3ad-406f-868f-cf855de3f492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.160520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd25e2c-c3ad-406f-868f-cf855de3f492 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.160797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd25e2c-c3ad-406f-868f-cf855de3f492 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.161067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd25e2c-c3ad-406f-868f-cf855de3f492 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.161700 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd25e2c-c3ad-406f-868f-cf855de3f492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.163363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd25e2c-c3ad-406f-868f-cf855de3f492 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.179776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 798/1592] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:26.180648 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd25e2c-c3ad-406f-868f-cf855de3f492 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.190304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd25e2c-c3ad-406f-868f-cf855de3f492 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.195697 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b907ca-352a-491b-87d5-3e48529ecccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.197635 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b907ca-352a-491b-87d5-3e48529ecccf admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.197635 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b907ca-352a-491b-87d5-3e48529ecccf admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.197635 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b907ca-352a-491b-87d5-3e48529ecccf admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.197635 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b907ca-352a-491b-87d5-3e48529ecccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.199223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b907ca-352a-491b-87d5-3e48529ecccf admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.210021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 799/1593] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:26.210881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b907ca-352a-491b-87d5-3e48529ecccf admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.220739 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 795/1594] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:26 2022] PUT /identity/v3/projects/af84072cdab4455a95655c3c5628e3ac/users/7dc5cd44b1b34a7ebbdca2c062715431/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:26.220935 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b907ca-352a-491b-87d5-3e48529ecccf admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.221800 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ee6b1-679e-4941-8cc2-826d2f434050 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.291973 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330eca1-9815-4ff6-ac6b-606353b00b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.292489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330eca1-9815-4ff6-ac6b-606353b00b66 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.292767 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330eca1-9815-4ff6-ac6b-606353b00b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.293040 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330eca1-9815-4ff6-ac6b-606353b00b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.293160 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ee6b1-679e-4941-8cc2-826d2f434050 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.293678 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330eca1-9815-4ff6-ac6b-606353b00b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.295302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330eca1-9815-4ff6-ac6b-606353b00b66 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.297893 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fd11c-5e57-4080-8aff-83aba6665f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.298604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fd11c-5e57-4080-8aff-83aba6665f86 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.298879 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fd11c-5e57-4080-8aff-83aba6665f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.299177 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fd11c-5e57-4080-8aff-83aba6665f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.299824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5fd11c-5e57-4080-8aff-83aba6665f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.301421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5fd11c-5e57-4080-8aff-83aba6665f86 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.305637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 800/1595] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:26.306498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330eca1-9815-4ff6-ac6b-606353b00b66 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.312402 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 796/1596] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:26.313077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fd11c-5e57-4080-8aff-83aba6665f86 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.316768 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330eca1-9815-4ff6-ac6b-606353b00b66 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.321669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015e0d4-34b6-4598-b3c0-afd4644e90fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.322251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015e0d4-34b6-4598-b3c0-afd4644e90fe admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.322525 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015e0d4-34b6-4598-b3c0-afd4644e90fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.322799 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015e0d4-34b6-4598-b3c0-afd4644e90fe admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.323349 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fd11c-5e57-4080-8aff-83aba6665f86 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.323464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015e0d4-34b6-4598-b3c0-afd4644e90fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.325027 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015e0d4-34b6-4598-b3c0-afd4644e90fe admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.328244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5658f72-427a-4455-af67-797e43a33fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.329013 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5658f72-427a-4455-af67-797e43a33fef admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.329284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5658f72-427a-4455-af67-797e43a33fef admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.329615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5658f72-427a-4455-af67-797e43a33fef admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.330443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5658f72-427a-4455-af67-797e43a33fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.331857 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5658f72-427a-4455-af67-797e43a33fef admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.337748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 801/1597] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:26.337748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015e0d4-34b6-4598-b3c0-afd4644e90fe admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.346526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 797/1598] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:26.346526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5658f72-427a-4455-af67-797e43a33fef admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.347596 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015e0d4-34b6-4598-b3c0-afd4644e90fe admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.352952 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73435f-5462-4ea8-ad6c-8d37f0c86b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.353556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73435f-5462-4ea8-ad6c-8d37f0c86b40 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.353863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73435f-5462-4ea8-ad6c-8d37f0c86b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.354133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73435f-5462-4ea8-ad6c-8d37f0c86b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.354730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5658f72-427a-4455-af67-797e43a33fef admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.354919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73435f-5462-4ea8-ad6c-8d37f0c86b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.356896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73435f-5462-4ea8-ad6c-8d37f0c86b40 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.360675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30915112-0b49-4b84-89f6-582d4bc6437c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.361426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30915112-0b49-4b84-89f6-582d4bc6437c admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.361744 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30915112-0b49-4b84-89f6-582d4bc6437c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.361958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30915112-0b49-4b84-89f6-582d4bc6437c admin admin] PATH_INFO: `/v3/projects/f1741ba78495437c8b24846da584f507/users/2034eed909584bcd9d4efeedca8c9719/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.368300 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 802/1599] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:26.369381 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee1aa2-f201-4f48-8ae1-abb2b2dde0a4 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.369671 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee1aa2-f201-4f48-8ae1-abb2b2dde0a4 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.369941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee1aa2-f201-4f48-8ae1-abb2b2dde0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.379610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30915112-0b49-4b84-89f6-582d4bc6437c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1741ba78495437c8b24846da584f507, user_id=2034eed909584bcd9d4efeedca8c9719, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.382571 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30915112-0b49-4b84-89f6-582d4bc6437c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.391124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-afee1aa2-f201-4f48-8ae1-abb2b2dde0a4 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:26.392365 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-afee1aa2-f201-4f48-8ae1-abb2b2dde0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:26.400379 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 803/1600] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:26.404664 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee1aa2-f201-4f48-8ae1-abb2b2dde0a4 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.407015 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:INSERT=9,keystone:SELECT=91 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:26.420689 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee1aa2-f201-4f48-8ae1-abb2b2dde0a4 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.426418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883fc101-b506-4571-91e7-38e564ea20ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.427661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fc101-b506-4571-91e7-38e564ea20ec admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.427661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fc101-b506-4571-91e7-38e564ea20ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.427661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fc101-b506-4571-91e7-38e564ea20ec admin admin] PATH_INFO: `/v3/projects/507c97f5d4a74853a3c50b4aef9e891c/users/3bd31b8807fc450f8f134417cd535542/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.443528 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883fc101-b506-4571-91e7-38e564ea20ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=507c97f5d4a74853a3c50b4aef9e891c, user_id=3bd31b8807fc450f8f134417cd535542, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.446201 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883fc101-b506-4571-91e7-38e564ea20ec admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.452255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 798/1601] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:26 2022] PUT /identity/v3/projects/f1741ba78495437c8b24846da584f507/users/2034eed909584bcd9d4efeedca8c9719/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:26.453263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30915112-0b49-4b84-89f6-582d4bc6437c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.458713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:26.517818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30915112-0b49-4b84-89f6-582d4bc6437c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.522460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879bd90-32ce-4f0d-9bd3-9bcbd27d1f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.523141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879bd90-32ce-4f0d-9bd3-9bcbd27d1f72 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.523421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879bd90-32ce-4f0d-9bd3-9bcbd27d1f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.523695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879bd90-32ce-4f0d-9bd3-9bcbd27d1f72 admin admin] PATH_INFO: `/v3/projects/fc121f6c44fb497399c6a7f34653066c/users/828c68b1ba3147cdb1659161178a5acc/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.533944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 804/1602] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:26 2022] PUT /identity/v3/projects/507c97f5d4a74853a3c50b4aef9e891c/users/3bd31b8807fc450f8f134417cd535542/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:26.538591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8879bd90-32ce-4f0d-9bd3-9bcbd27d1f72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc121f6c44fb497399c6a7f34653066c, user_id=828c68b1ba3147cdb1659161178a5acc, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.539137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883fc101-b506-4571-91e7-38e564ea20ec admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.541361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8879bd90-32ce-4f0d-9bd3-9bcbd27d1f72 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.599955 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883fc101-b506-4571-91e7-38e564ea20ec admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.605291 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a7164-cc9f-4ae5-9ee8-28ad8618ddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.605990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a7164-cc9f-4ae5-9ee8-28ad8618ddad admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.606269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a7164-cc9f-4ae5-9ee8-28ad8618ddad admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.606567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a7164-cc9f-4ae5-9ee8-28ad8618ddad admin admin] PATH_INFO: `/v3/projects/253dee88af3348949a17a4ff0dcc69de/users/128278a80e304e94b9741a337d8040c0/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.620423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 799/1603] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:26 2022] PUT /identity/v3/projects/fc121f6c44fb497399c6a7f34653066c/users/828c68b1ba3147cdb1659161178a5acc/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:26.620423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879bd90-32ce-4f0d-9bd3-9bcbd27d1f72 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.624037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a7164-cc9f-4ae5-9ee8-28ad8618ddad admin admin] RBAC: Authorizing `identity:create_grant(project_id=253dee88af3348949a17a4ff0dcc69de, user_id=128278a80e304e94b9741a337d8040c0, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.627237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a7164-cc9f-4ae5-9ee8-28ad8618ddad admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.686946 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879bd90-32ce-4f0d-9bd3-9bcbd27d1f72 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.691636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede93ae-024d-4c3c-85e6-91a967b1d257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.692298 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede93ae-024d-4c3c-85e6-91a967b1d257 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.692645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede93ae-024d-4c3c-85e6-91a967b1d257 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.692938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede93ae-024d-4c3c-85e6-91a967b1d257 admin admin] PATH_INFO: `/v3/projects/92b4ddd948934014bd572c1e9a4800e5/users/3f3e15e7a6354ca99e8635a0948af26d/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.710103 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 805/1604] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:26 2022] PUT /identity/v3/projects/253dee88af3348949a17a4ff0dcc69de/users/128278a80e304e94b9741a337d8040c0/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:26.722461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a7164-cc9f-4ae5-9ee8-28ad8618ddad admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.731249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede93ae-024d-4c3c-85e6-91a967b1d257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92b4ddd948934014bd572c1e9a4800e5, user_id=3f3e15e7a6354ca99e8635a0948af26d, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.734313 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede93ae-024d-4c3c-85e6-91a967b1d257 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.788318 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a7164-cc9f-4ae5-9ee8-28ad8618ddad admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.792894 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f496078-6e2f-4bb7-89f2-85499ca36510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.793539 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f496078-6e2f-4bb7-89f2-85499ca36510 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.793811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f496078-6e2f-4bb7-89f2-85499ca36510 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.794077 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f496078-6e2f-4bb7-89f2-85499ca36510 admin admin] PATH_INFO: `/v3/projects/d33a78463eba4967826c9917dd352db2/users/39d98294ba884280b4971af36a9fbd6e/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.809542 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f496078-6e2f-4bb7-89f2-85499ca36510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d33a78463eba4967826c9917dd352db2, user_id=39d98294ba884280b4971af36a9fbd6e, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.812298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f496078-6e2f-4bb7-89f2-85499ca36510 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.823842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 800/1605] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:26 2022] PUT /identity/v3/projects/92b4ddd948934014bd572c1e9a4800e5/users/3f3e15e7a6354ca99e8635a0948af26d/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:26.825038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede93ae-024d-4c3c-85e6-91a967b1d257 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:26.889587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede93ae-024d-4c3c-85e6-91a967b1d257 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:26.893343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17c46a-1e6c-417f-9e47-247745114153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:26.893951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17c46a-1e6c-417f-9e47-247745114153 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.894233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17c46a-1e6c-417f-9e47-247745114153 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.894570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17c46a-1e6c-417f-9e47-247745114153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.918517 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:26.923329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 806/1606] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:26 2022] PUT /identity/v3/projects/d33a78463eba4967826c9917dd352db2/users/39d98294ba884280b4971af36a9fbd6e/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:26.923329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490826e4-61e8-49b3-a907-681b35f7d4f3 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.923329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490826e4-61e8-49b3-a907-681b35f7d4f3 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.923329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490826e4-61e8-49b3-a907-681b35f7d4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.952022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-490826e4-61e8-49b3-a907-681b35f7d4f3 None None] MFA Rules not processed for user `2034eed909584bcd9d4efeedca8c9719`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:26.954060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-490826e4-61e8-49b3-a907-681b35f7d4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:26.966460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17c46a-1e6c-417f-9e47-247745114153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:26.967952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17c46a-1e6c-417f-9e47-247745114153 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:26.978379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 801/1607] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:26.979713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199c4de-db6e-459d-b8d0-57e2abcfcf7f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:26.980007 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199c4de-db6e-459d-b8d0-57e2abcfcf7f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:26.980347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199c4de-db6e-459d-b8d0-57e2abcfcf7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:26.995516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:26.999989 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a199c4de-db6e-459d-b8d0-57e2abcfcf7f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:27.001199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a199c4de-db6e-459d-b8d0-57e2abcfcf7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:27.030428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 807/1608] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:26 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:27.032690 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfa860-14f9-4dfa-aafa-7bb2671589ff None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.033661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfa860-14f9-4dfa-aafa-7bb2671589ff None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.034154 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfa860-14f9-4dfa-aafa-7bb2671589ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.064398 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-a5bfa860-14f9-4dfa-aafa-7bb2671589ff None None] MFA Rules not processed for user `828c68b1ba3147cdb1659161178a5acc`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:27.065531 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a5bfa860-14f9-4dfa-aafa-7bb2671589ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:27.067792 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 802/1609] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:27.071316 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc2523-1940-464b-b224-1e0db20177b7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.071316 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc2523-1940-464b-b224-1e0db20177b7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.071316 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc2523-1940-464b-b224-1e0db20177b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.106866 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e0cc2523-1940-464b-b224-1e0db20177b7 None None] MFA Rules not processed for user `128278a80e304e94b9741a337d8040c0`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:27.111424 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e0cc2523-1940-464b-b224-1e0db20177b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:27.147822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 808/1610] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:27.147822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449631b-f4cc-4ee7-8f67-631ede23706a None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.147822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449631b-f4cc-4ee7-8f67-631ede23706a None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.147822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449631b-f4cc-4ee7-8f67-631ede23706a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.178835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-f449631b-f4cc-4ee7-8f67-631ede23706a None None] MFA Rules not processed for user `3f3e15e7a6354ca99e8635a0948af26d`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:27.180168 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-f449631b-f4cc-4ee7-8f67-631ede23706a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:27.196188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 803/1611] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:27.199885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c19b58-9dbb-40d8-b2ec-3c4b714c6cea None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.199885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c19b58-9dbb-40d8-b2ec-3c4b714c6cea None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.199885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c19b58-9dbb-40d8-b2ec-3c4b714c6cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.218939 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-66c19b58-9dbb-40d8-b2ec-3c4b714c6cea None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:27.220188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-66c19b58-9dbb-40d8-b2ec-3c4b714c6cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:27.230536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 804/1612] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:27.235463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c19b58-9dbb-40d8-b2ec-3c4b714c6cea None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.247896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c19b58-9dbb-40d8-b2ec-3c4b714c6cea None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.252440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63902e-54a0-4cea-b6f9-f0717fcd893d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.253026 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63902e-54a0-4cea-b6f9-f0717fcd893d admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.257361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63902e-54a0-4cea-b6f9-f0717fcd893d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.257361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63902e-54a0-4cea-b6f9-f0717fcd893d admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.257361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63902e-54a0-4cea-b6f9-f0717fcd893d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.257361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63902e-54a0-4cea-b6f9-f0717fcd893d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.259730 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 809/1613] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:27.264090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449631b-f4cc-4ee7-8f67-631ede23706a None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.320234 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449631b-f4cc-4ee7-8f67-631ede23706a None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.323919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8f70b-dfd6-46a4-90a0-b36a982ec815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.324523 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8f70b-dfd6-46a4-90a0-b36a982ec815 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.324805 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8f70b-dfd6-46a4-90a0-b36a982ec815 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.325076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8f70b-dfd6-46a4-90a0-b36a982ec815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.335148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8f70b-dfd6-46a4-90a0-b36a982ec815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.336621 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8f70b-dfd6-46a4-90a0-b36a982ec815 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.346908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 810/1614] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:27.356399 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8f70b-dfd6-46a4-90a0-b36a982ec815 service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.368364 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8f70b-dfd6-46a4-90a0-b36a982ec815 service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.376220 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4261aaf-1f55-41df-bf01-52e43323d2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.376885 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4261aaf-1f55-41df-bf01-52e43323d2c0 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.377114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4261aaf-1f55-41df-bf01-52e43323d2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.377418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4261aaf-1f55-41df-bf01-52e43323d2c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.378058 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4261aaf-1f55-41df-bf01-52e43323d2c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.380324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4261aaf-1f55-41df-bf01-52e43323d2c0 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.383243 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 805/1615] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:27.384428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63902e-54a0-4cea-b6f9-f0717fcd893d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.402826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:27.410121 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:27.454966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63902e-54a0-4cea-b6f9-f0717fcd893d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.460463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddcb6f1-bc20-48fc-b37d-6b4148693b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.461204 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcb6f1-bc20-48fc-b37d-6b4148693b8b admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.461693 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcb6f1-bc20-48fc-b37d-6b4148693b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.462063 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcb6f1-bc20-48fc-b37d-6b4148693b8b admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.462823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddcb6f1-bc20-48fc-b37d-6b4148693b8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.465066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddcb6f1-bc20-48fc-b37d-6b4148693b8b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.518443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 811/1616] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:27.519648 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4261aaf-1f55-41df-bf01-52e43323d2c0 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.574402 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 806/1617] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/projects => generated 355 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:27.577186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8ad26-3cb0-459b-8d80-52a7408e5676 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.579943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8ad26-3cb0-459b-8d80-52a7408e5676 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.580266 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8ad26-3cb0-459b-8d80-52a7408e5676 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.583956 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 807/1618] 10.222.0.27 () {54 vars in 962 bytes} [Tue Aug 30 14:56:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:56:27.585214 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8ad26-3cb0-459b-8d80-52a7408e5676 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.621358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4261aaf-1f55-41df-bf01-52e43323d2c0 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.664723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8ad26-3cb0-459b-8d80-52a7408e5676 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.670239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a143cb-daa9-4eff-944f-5d68d294cdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.670847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a143cb-daa9-4eff-944f-5d68d294cdbe admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.671146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a143cb-daa9-4eff-944f-5d68d294cdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.671440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a143cb-daa9-4eff-944f-5d68d294cdbe admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.672090 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a143cb-daa9-4eff-944f-5d68d294cdbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.674056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a143cb-daa9-4eff-944f-5d68d294cdbe admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.678516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8afbcb-7c84-47b3-bc00-54ad8869bbcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.679312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8afbcb-7c84-47b3-bc00-54ad8869bbcc service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.679661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8afbcb-7c84-47b3-bc00-54ad8869bbcc service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.680090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8afbcb-7c84-47b3-bc00-54ad8869bbcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.689737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8afbcb-7c84-47b3-bc00-54ad8869bbcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.691339 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8afbcb-7c84-47b3-bc00-54ad8869bbcc service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.701677 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 812/1619] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:27.702646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8afbcb-7c84-47b3-bc00-54ad8869bbcc service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.713082 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8afbcb-7c84-47b3-bc00-54ad8869bbcc service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.718356 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df8ce5-d3fe-4876-877e-9a2ec8e24392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.719266 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df8ce5-d3fe-4876-877e-9a2ec8e24392 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.719518 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df8ce5-d3fe-4876-877e-9a2ec8e24392 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.719849 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df8ce5-d3fe-4876-877e-9a2ec8e24392 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.720649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df8ce5-d3fe-4876-877e-9a2ec8e24392 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.722882 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df8ce5-d3fe-4876-877e-9a2ec8e24392 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.753957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 808/1620] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/users => generated 309 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:27.758358 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a143cb-daa9-4eff-944f-5d68d294cdbe admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.769600 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a143cb-daa9-4eff-944f-5d68d294cdbe admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.776705 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a0c68-abf6-4b42-9cde-868bb5051c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.777610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a0c68-abf6-4b42-9cde-868bb5051c6a admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.777899 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a0c68-abf6-4b42-9cde-868bb5051c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.778170 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a0c68-abf6-4b42-9cde-868bb5051c6a admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.778815 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a0c68-abf6-4b42-9cde-868bb5051c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.780847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a0c68-abf6-4b42-9cde-868bb5051c6a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.832614 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 813/1621] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/users => generated 300 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:27.835554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7798877-6b00-4dc3-9b65-21d979388a24 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.836099 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7798877-6b00-4dc3-9b65-21d979388a24 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.836616 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7798877-6b00-4dc3-9b65-21d979388a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.863896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 809/1622] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/users => generated 284 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:27.868348 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a0c68-abf6-4b42-9cde-868bb5051c6a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.880972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a0c68-abf6-4b42-9cde-868bb5051c6a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.884035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e7798877-6b00-4dc3-9b65-21d979388a24 None None] MFA Rules not processed for user `5b6e4c5bf2a54b28a55e5ae74b90ab4f`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:27.885332 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e7798877-6b00-4dc3-9b65-21d979388a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:27.888279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2d6cc-5a08-465c-9cf7-892082b5a4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.889108 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2d6cc-5a08-465c-9cf7-892082b5a4fe admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.889733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2d6cc-5a08-465c-9cf7-892082b5a4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.890153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2d6cc-5a08-465c-9cf7-892082b5a4fe admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.891019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2d6cc-5a08-465c-9cf7-892082b5a4fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.894135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2d6cc-5a08-465c-9cf7-892082b5a4fe admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.906354 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 810/1623] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:27.908204 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2d6cc-5a08-465c-9cf7-892082b5a4fe admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.920175 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2d6cc-5a08-465c-9cf7-892082b5a4fe admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.926131 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ec418-668e-4857-a5b3-f37d00d82811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.927007 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ec418-668e-4857-a5b3-f37d00d82811 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.935431 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ec418-668e-4857-a5b3-f37d00d82811 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.935431 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ec418-668e-4857-a5b3-f37d00d82811 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.935431 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ec418-668e-4857-a5b3-f37d00d82811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.935431 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ec418-668e-4857-a5b3-f37d00d82811 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.947306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 811/1624] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:27.948748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ec418-668e-4857-a5b3-f37d00d82811 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:27.963494 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ec418-668e-4857-a5b3-f37d00d82811 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:27.969199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282249a-ec87-4515-b349-7f52d2e1bf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:27.970055 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282249a-ec87-4515-b349-7f52d2e1bf89 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:27.971072 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282249a-ec87-4515-b349-7f52d2e1bf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:27.971602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282249a-ec87-4515-b349-7f52d2e1bf89 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:27.972400 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282249a-ec87-4515-b349-7f52d2e1bf89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:27.974228 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282249a-ec87-4515-b349-7f52d2e1bf89 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:27.988252 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 812/1625] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:27.989696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282249a-ec87-4515-b349-7f52d2e1bf89 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:28.001809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282249a-ec87-4515-b349-7f52d2e1bf89 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:28.002311 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:28.008573 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6512f9-9901-495b-bb8e-5a7a8b6c1571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:28.009930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6512f9-9901-495b-bb8e-5a7a8b6c1571 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.015981 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6512f9-9901-495b-bb8e-5a7a8b6c1571 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.015981 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6512f9-9901-495b-bb8e-5a7a8b6c1571 admin admin] PATH_INFO: `/v3/projects/8df76135f0a84ffbaf2887686f084463/users/cbe21d7e447949668d1081c7e1ae0dfe/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.031641 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 814/1626] 10.222.0.27 () {58 vars in 1059 bytes} [Tue Aug 30 14:56:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:28.032398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6512f9-9901-495b-bb8e-5a7a8b6c1571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8df76135f0a84ffbaf2887686f084463, user_id=cbe21d7e447949668d1081c7e1ae0dfe, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:28.035971 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6512f9-9901-495b-bb8e-5a7a8b6c1571 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:28.040021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7798877-6b00-4dc3-9b65-21d979388a24 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:28.055983 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7798877-6b00-4dc3-9b65-21d979388a24 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:28.062839 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105726a0-5c7a-46f7-b653-62290a3f770d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:28.064901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105726a0-5c7a-46f7-b653-62290a3f770d admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.066082 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105726a0-5c7a-46f7-b653-62290a3f770d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.066619 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105726a0-5c7a-46f7-b653-62290a3f770d admin admin] PATH_INFO: `/v3/projects/2b71357a6c004abe891759173d465629/users/463ffe94e0de477f9d5941ed1c375174/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.092329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105726a0-5c7a-46f7-b653-62290a3f770d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b71357a6c004abe891759173d465629, user_id=463ffe94e0de477f9d5941ed1c375174, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:28.095289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105726a0-5c7a-46f7-b653-62290a3f770d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:28.137900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 813/1627] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:27 2022] PUT /identity/v3/projects/8df76135f0a84ffbaf2887686f084463/users/cbe21d7e447949668d1081c7e1ae0dfe/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:28.141523 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6512f9-9901-495b-bb8e-5a7a8b6c1571 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:28.192563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 815/1628] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:28 2022] PUT /identity/v3/projects/2b71357a6c004abe891759173d465629/users/463ffe94e0de477f9d5941ed1c375174/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:28.195951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105726a0-5c7a-46f7-b653-62290a3f770d admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:28.211336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6512f9-9901-495b-bb8e-5a7a8b6c1571 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:28.283294 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac9ddd-eea1-4be6-bb8a-eb96de0dd2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:28.283973 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac9ddd-eea1-4be6-bb8a-eb96de0dd2c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.284255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac9ddd-eea1-4be6-bb8a-eb96de0dd2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.284528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac9ddd-eea1-4be6-bb8a-eb96de0dd2c4 admin admin] PATH_INFO: `/v3/projects/155748f5300142d396c4724884a4897b/users/ce515bc9fbeb4be78038c92fbaa71ba5/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.301520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac9ddd-eea1-4be6-bb8a-eb96de0dd2c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=155748f5300142d396c4724884a4897b, user_id=ce515bc9fbeb4be78038c92fbaa71ba5, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:28.301744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105726a0-5c7a-46f7-b653-62290a3f770d admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:28.305039 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac9ddd-eea1-4be6-bb8a-eb96de0dd2c4 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:28.311120 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ee1e2-7d63-4bb1-adab-e0b13bb0a3a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:28.311928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ee1e2-7d63-4bb1-adab-e0b13bb0a3a8 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.312667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ee1e2-7d63-4bb1-adab-e0b13bb0a3a8 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.312667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ee1e2-7d63-4bb1-adab-e0b13bb0a3a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.417683 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 814/1629] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:28 2022] PUT /identity/v3/projects/155748f5300142d396c4724884a4897b/users/ce515bc9fbeb4be78038c92fbaa71ba5/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:28.418634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac9ddd-eea1-4be6-bb8a-eb96de0dd2c4 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:28.432323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ee1e2-7d63-4bb1-adab-e0b13bb0a3a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:28.433976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ee1e2-7d63-4bb1-adab-e0b13bb0a3a8 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:28.441495 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:28.446584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 816/1630] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:28.447395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ee1e2-7d63-4bb1-adab-e0b13bb0a3a8 service nova] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:28.460093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ee1e2-7d63-4bb1-adab-e0b13bb0a3a8 service nova] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:28.535240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5466e3-5454-435c-86dd-1dced39e1369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:28.536847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5466e3-5454-435c-86dd-1dced39e1369 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.537876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5466e3-5454-435c-86dd-1dced39e1369 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.538434 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5466e3-5454-435c-86dd-1dced39e1369 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.539302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5466e3-5454-435c-86dd-1dced39e1369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:28.541503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5466e3-5454-435c-86dd-1dced39e1369 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:28.553853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac9ddd-eea1-4be6-bb8a-eb96de0dd2c4 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:28.567360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2477fa-3c20-4cdc-8d52-24c1236fc82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:28.568002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2477fa-3c20-4cdc-8d52-24c1236fc82d admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.568291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2477fa-3c20-4cdc-8d52-24c1236fc82d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.568601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2477fa-3c20-4cdc-8d52-24c1236fc82d admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.571332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2477fa-3c20-4cdc-8d52-24c1236fc82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:28.574232 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2477fa-3c20-4cdc-8d52-24c1236fc82d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:28.580696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 817/1631] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:28.581581 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5466e3-5454-435c-86dd-1dced39e1369 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:28.600676 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 815/1632] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:28.601552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2477fa-3c20-4cdc-8d52-24c1236fc82d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:28.606638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5466e3-5454-435c-86dd-1dced39e1369 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:28.620296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ceee1-089f-42a4-bd5a-ebf063f9d60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:28.621088 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ceee1-089f-42a4-bd5a-ebf063f9d60b admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.621423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ceee1-089f-42a4-bd5a-ebf063f9d60b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.621698 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ceee1-089f-42a4-bd5a-ebf063f9d60b admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.622350 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ceee1-089f-42a4-bd5a-ebf063f9d60b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:28.625879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ceee1-089f-42a4-bd5a-ebf063f9d60b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:28.626451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2477fa-3c20-4cdc-8d52-24c1236fc82d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:28.638653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97f9ed-458a-4506-afec-eff7b6e32683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:28.639363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97f9ed-458a-4506-afec-eff7b6e32683 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.639645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97f9ed-458a-4506-afec-eff7b6e32683 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.639924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97f9ed-458a-4506-afec-eff7b6e32683 admin admin] PATH_INFO: `/v3/projects/2b71357a6c004abe891759173d465629/users/463ffe94e0de477f9d5941ed1c375174/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.648690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97f9ed-458a-4506-afec-eff7b6e32683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b71357a6c004abe891759173d465629, user_id=463ffe94e0de477f9d5941ed1c375174, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:28.652896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 818/1633] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:28.652896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ceee1-089f-42a4-bd5a-ebf063f9d60b admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:28.653101 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97f9ed-458a-4506-afec-eff7b6e32683 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:28.672210 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ceee1-089f-42a4-bd5a-ebf063f9d60b admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:28.684315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 816/1634] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:28 2022] PUT /identity/v3/projects/2b71357a6c004abe891759173d465629/users/463ffe94e0de477f9d5941ed1c375174/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:28.684315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97f9ed-458a-4506-afec-eff7b6e32683 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:28.737477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:28.765894 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c09d1f-15fc-4b18-9d4e-e44478fc1758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:28.769902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c09d1f-15fc-4b18-9d4e-e44478fc1758 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.770599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c09d1f-15fc-4b18-9d4e-e44478fc1758 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.771222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c09d1f-15fc-4b18-9d4e-e44478fc1758 admin admin] PATH_INFO: `/v3/projects/8df76135f0a84ffbaf2887686f084463/users/cbe21d7e447949668d1081c7e1ae0dfe/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.773982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c09d1f-15fc-4b18-9d4e-e44478fc1758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8df76135f0a84ffbaf2887686f084463, user_id=cbe21d7e447949668d1081c7e1ae0dfe, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:28.777203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c09d1f-15fc-4b18-9d4e-e44478fc1758 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:28.790252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 819/1635] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:28 2022] PUT /identity/v3/projects/8df76135f0a84ffbaf2887686f084463/users/cbe21d7e447949668d1081c7e1ae0dfe/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:28.791872 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d7078-9eab-4bd3-ac04-0a238f2e8caf None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.792256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d7078-9eab-4bd3-ac04-0a238f2e8caf None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.792665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d7078-9eab-4bd3-ac04-0a238f2e8caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.813477 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97f9ed-458a-4506-afec-eff7b6e32683 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:28.831574 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-6a6d7078-9eab-4bd3-ac04-0a238f2e8caf None None] MFA Rules not processed for user `463ffe94e0de477f9d5941ed1c375174`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:28.833362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6a6d7078-9eab-4bd3-ac04-0a238f2e8caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:28.910079 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d15fb-4527-496b-9fa0-dc219c0f3b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:28.910921 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d15fb-4527-496b-9fa0-dc219c0f3b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.911303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d15fb-4527-496b-9fa0-dc219c0f3b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.911629 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d15fb-4527-496b-9fa0-dc219c0f3b44 admin admin] PATH_INFO: `/v3/projects/155748f5300142d396c4724884a4897b/users/ce515bc9fbeb4be78038c92fbaa71ba5/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.913813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d15fb-4527-496b-9fa0-dc219c0f3b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=155748f5300142d396c4724884a4897b, user_id=ce515bc9fbeb4be78038c92fbaa71ba5, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:28.916910 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d15fb-4527-496b-9fa0-dc219c0f3b44 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:28.940622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 817/1636] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:28 2022] PUT /identity/v3/projects/155748f5300142d396c4724884a4897b/users/ce515bc9fbeb4be78038c92fbaa71ba5/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:28.940622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcdf79-da2d-4462-b787-e727e9098fd6 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.940622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcdf79-da2d-4462-b787-e727e9098fd6 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.940622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcdf79-da2d-4462-b787-e727e9098fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.971234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-64fcdf79-da2d-4462-b787-e727e9098fd6 None None] MFA Rules not processed for user `cbe21d7e447949668d1081c7e1ae0dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:28.973131 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-64fcdf79-da2d-4462-b787-e727e9098fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:28.994592 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 820/1637] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:28 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:28.994592 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bf19e-e09a-4c32-b6fc-56804d71c54b None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.994592 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bf19e-e09a-4c32-b6fc-56804d71c54b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.994592 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bf19e-e09a-4c32-b6fc-56804d71c54b None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:28.996263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 821/1638] 10.222.0.27 () {54 vars in 912 bytes} [Tue Aug 30 14:56:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:56:28.997669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7f47a-a0aa-4bc7-8477-1e056a507d16 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:28.998073 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7f47a-a0aa-4bc7-8477-1e056a507d16 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:28.998448 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7f47a-a0aa-4bc7-8477-1e056a507d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.040137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e1e7f47a-a0aa-4bc7-8477-1e056a507d16 None None] MFA Rules not processed for user `ce515bc9fbeb4be78038c92fbaa71ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:29.042243 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e1e7f47a-a0aa-4bc7-8477-1e056a507d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:29.095754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 818/1639] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:28 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:29.097715 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8d557-3fc3-4112-8af3-9332601d61e0 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:29.098167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8d557-3fc3-4112-8af3-9332601d61e0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:29.098436 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8d557-3fc3-4112-8af3-9332601d61e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.133664 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-79d8d557-3fc3-4112-8af3-9332601d61e0 None None] MFA Rules not processed for user `5b6e4c5bf2a54b28a55e5ae74b90ab4f`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:29.136293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-79d8d557-3fc3-4112-8af3-9332601d61e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:29.168862 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 822/1640] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:28 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:29.170131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7f47a-a0aa-4bc7-8477-1e056a507d16 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:29.283924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 819/1641] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 30 14:56:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:29.284942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8d557-3fc3-4112-8af3-9332601d61e0 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:29.394285 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7f47a-a0aa-4bc7-8477-1e056a507d16 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:29.394285 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77da734-6f4b-4df6-b708-b7dca68b0337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:29.394285 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77da734-6f4b-4df6-b708-b7dca68b0337 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:29.394285 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77da734-6f4b-4df6-b708-b7dca68b0337 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:29.394863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77da734-6f4b-4df6-b708-b7dca68b0337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.430473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8d557-3fc3-4112-8af3-9332601d61e0 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:29.441313 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9205fa5-4db9-41a8-8ddc-004323ebffaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:29.441943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9205fa5-4db9-41a8-8ddc-004323ebffaf service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:29.442227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9205fa5-4db9-41a8-8ddc-004323ebffaf service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:29.442502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9205fa5-4db9-41a8-8ddc-004323ebffaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.458361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9205fa5-4db9-41a8-8ddc-004323ebffaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:29.459943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9205fa5-4db9-41a8-8ddc-004323ebffaf service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:29.479837 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:29.480844 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 820/1642] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:29 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:29.482851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77da734-6f4b-4df6-b708-b7dca68b0337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:29.484443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77da734-6f4b-4df6-b708-b7dca68b0337 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:29.511873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 823/1643] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:29.571629 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9205fa5-4db9-41a8-8ddc-004323ebffaf service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:29.582509 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9205fa5-4db9-41a8-8ddc-004323ebffaf service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:29.596801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77da734-6f4b-4df6-b708-b7dca68b0337 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:29.601370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433c60c-1b16-4db7-a242-af91c72191ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:29.601991 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433c60c-1b16-4db7-a242-af91c72191ba service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:29.606304 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433c60c-1b16-4db7-a242-af91c72191ba service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:29.606540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433c60c-1b16-4db7-a242-af91c72191ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.608939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77da734-6f4b-4df6-b708-b7dca68b0337 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:29.621369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcf40d-6848-409c-a499-12a628701d29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:29.622022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcf40d-6848-409c-a499-12a628701d29 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:29.622307 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcf40d-6848-409c-a499-12a628701d29 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:29.622587 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcf40d-6848-409c-a499-12a628701d29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:29.632134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433c60c-1b16-4db7-a242-af91c72191ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:29.633777 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433c60c-1b16-4db7-a242-af91c72191ba service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:29.650367 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fcf40d-6848-409c-a499-12a628701d29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:29.652015 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fcf40d-6848-409c-a499-12a628701d29 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:29.677093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 821/1644] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:29 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:29.692851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 824/1645] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:30.259395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:30.309335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433c60c-1b16-4db7-a242-af91c72191ba service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:30.321164 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433c60c-1b16-4db7-a242-af91c72191ba service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:30.332816 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81863d-cfc8-4c78-91b2-665cb302cea5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:30.333474 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81863d-cfc8-4c78-91b2-665cb302cea5 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:30.333905 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81863d-cfc8-4c78-91b2-665cb302cea5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:30.337161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81863d-cfc8-4c78-91b2-665cb302cea5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:30.453095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81863d-cfc8-4c78-91b2-665cb302cea5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:30.454642 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81863d-cfc8-4c78-91b2-665cb302cea5 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:30.466307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 822/1646] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:30 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:31.095175 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:31.231875 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcf40d-6848-409c-a499-12a628701d29 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:31.487953 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81863d-cfc8-4c78-91b2-665cb302cea5 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:31.504829 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcf40d-6848-409c-a499-12a628701d29 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:31.530196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be3c7b-000a-47a6-a133-e8a77eb1327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:31.531033 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be3c7b-000a-47a6-a133-e8a77eb1327e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:31.531682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81863d-cfc8-4c78-91b2-665cb302cea5 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:31.532788 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be3c7b-000a-47a6-a133-e8a77eb1327e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:31.533467 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be3c7b-000a-47a6-a133-e8a77eb1327e admin admin] PATH_INFO: `/v3/users/2034eed909584bcd9d4efeedca8c9719` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:31.546423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be3c7b-000a-47a6-a133-e8a77eb1327e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2034eed909584bcd9d4efeedca8c9719)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:31.554037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be3c7b-000a-47a6-a133-e8a77eb1327e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:31.561208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb458b5-57a5-438f-8a5e-6140e2d05c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:31.561868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb458b5-57a5-438f-8a5e-6140e2d05c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:31.562154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb458b5-57a5-438f-8a5e-6140e2d05c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:31.562423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb458b5-57a5-438f-8a5e-6140e2d05c71 admin admin] PATH_INFO: `/v3/users/828c68b1ba3147cdb1659161178a5acc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:31.563633 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb458b5-57a5-438f-8a5e-6140e2d05c71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=828c68b1ba3147cdb1659161178a5acc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:31.564664 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: WARNING py.warnings [None req-6bb458b5-57a5-438f-8a5e-6140e2d05c71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:56:31.564664 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: warnings.warn(msg) Aug 30 14:56:31.564664 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: Aug 30 14:56:31.565912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb458b5-57a5-438f-8a5e-6140e2d05c71 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:31.863718 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:31.882644 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b4be3c7b-000a-47a6-a133-e8a77eb1327e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2034eed909584bcd9d4efeedca8c9719'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:31.903538 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b4be3c7b-000a-47a6-a133-e8a77eb1327e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2034eed909584bcd9d4efeedca8c9719'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:31.908636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6bb458b5-57a5-438f-8a5e-6140e2d05c71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '828c68b1ba3147cdb1659161178a5acc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:31.936821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b4be3c7b-000a-47a6-a133-e8a77eb1327e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2034eed909584bcd9d4efeedca8c9719'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:31.938107 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b4be3c7b-000a-47a6-a133-e8a77eb1327e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2034eed909584bcd9d4efeedca8c9719'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:31.938514 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b4be3c7b-000a-47a6-a133-e8a77eb1327e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2034eed909584bcd9d4efeedca8c9719'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:31.963478 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6bb458b5-57a5-438f-8a5e-6140e2d05c71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '828c68b1ba3147cdb1659161178a5acc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:31.967733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6bb458b5-57a5-438f-8a5e-6140e2d05c71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '828c68b1ba3147cdb1659161178a5acc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:32.039627 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 825/1647] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:31 2022] DELETE /identity/v3/users/2034eed909584bcd9d4efeedca8c9719 => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:32.041098 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5bafb-fd57-4355-bf09-212a1de7bb87 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:32.041414 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5bafb-fd57-4355-bf09-212a1de7bb87 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:32.041722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5bafb-fd57-4355-bf09-212a1de7bb87 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:32.049223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 826/1648] 10.222.0.27 () {54 vars in 962 bytes} [Tue Aug 30 14:56:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:56:32.050686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e487d22-4ecb-4548-a8a3-f8c1abfa8756 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:32.050975 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e487d22-4ecb-4548-a8a3-f8c1abfa8756 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:32.051275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e487d22-4ecb-4548-a8a3-f8c1abfa8756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:32.075132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-9e487d22-4ecb-4548-a8a3-f8c1abfa8756 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:32.076637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9e487d22-4ecb-4548-a8a3-f8c1abfa8756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:32.082561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6bb458b5-57a5-438f-8a5e-6140e2d05c71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '828c68b1ba3147cdb1659161178a5acc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:32.120416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6bb458b5-57a5-438f-8a5e-6140e2d05c71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '828c68b1ba3147cdb1659161178a5acc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:32.133404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 823/1649] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:31 2022] DELETE /identity/v3/users/828c68b1ba3147cdb1659161178a5acc => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:32.134858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504462b-07de-48c3-87eb-31e8cae682e7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:32.135192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504462b-07de-48c3-87eb-31e8cae682e7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:32.135471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504462b-07de-48c3-87eb-31e8cae682e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:32.168552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:32.172468 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 827/1650] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:32.173995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06a3a1-6bb1-4478-8cd1-d367dabaf26e None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:32.174276 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06a3a1-6bb1-4478-8cd1-d367dabaf26e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:32.174553 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06a3a1-6bb1-4478-8cd1-d367dabaf26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:32.181774 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e504462b-07de-48c3-87eb-31e8cae682e7 None None] MFA Rules not processed for user `5b6e4c5bf2a54b28a55e5ae74b90ab4f`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:32.183015 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e504462b-07de-48c3-87eb-31e8cae682e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:32.213943 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-7d06a3a1-6bb1-4478-8cd1-d367dabaf26e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:32.222928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7d06a3a1-6bb1-4478-8cd1-d367dabaf26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:32.326749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 828/1651] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:32.336073 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06a3a1-6bb1-4478-8cd1-d367dabaf26e None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:32.337876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7d06a3a1-6bb1-4478-8cd1-d367dabaf26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:32.364515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 824/1652] 10.222.0.27 () {58 vars in 1059 bytes} [Tue Aug 30 14:56:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:32.371209 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cfdf7-c39e-4e74-8eca-84ff5a43b9ee None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:32.371536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cfdf7-c39e-4e74-8eca-84ff5a43b9ee None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:32.371829 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cfdf7-c39e-4e74-8eca-84ff5a43b9ee None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:32.373163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 825/1653] 10.222.0.27 () {54 vars in 912 bytes} [Tue Aug 30 14:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:56:32.379262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af8cbe-b912-4ff2-9453-9b60afd8118d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:32.379757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af8cbe-b912-4ff2-9453-9b60afd8118d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:32.383685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af8cbe-b912-4ff2-9453-9b60afd8118d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:32.425057 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-48af8cbe-b912-4ff2-9453-9b60afd8118d None None] MFA Rules not processed for user `5b6e4c5bf2a54b28a55e5ae74b90ab4f`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:32.426483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-48af8cbe-b912-4ff2-9453-9b60afd8118d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:32.440325 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06a3a1-6bb1-4478-8cd1-d367dabaf26e None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:32.446323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684dfbea-f4fb-4a87-9f63-26815374dbb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:32.448119 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684dfbea-f4fb-4a87-9f63-26815374dbb0 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:32.448119 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684dfbea-f4fb-4a87-9f63-26815374dbb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:32.449969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684dfbea-f4fb-4a87-9f63-26815374dbb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:32.456121 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 826/1654] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 30 14:56:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:56:32.456857 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af8cbe-b912-4ff2-9453-9b60afd8118d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:32.458212 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-48af8cbe-b912-4ff2-9453-9b60afd8118d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:32.484703 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af8cbe-b912-4ff2-9453-9b60afd8118d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:32.497072 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cb172-2c2b-4d87-a23d-4dfa2557aeaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:32.497700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cb172-2c2b-4d87-a23d-4dfa2557aeaa service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:32.497980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cb172-2c2b-4d87-a23d-4dfa2557aeaa service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:32.498247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cb172-2c2b-4d87-a23d-4dfa2557aeaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:32.501191 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684dfbea-f4fb-4a87-9f63-26815374dbb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:32.505997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684dfbea-f4fb-4a87-9f63-26815374dbb0 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:32.519635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cb172-2c2b-4d87-a23d-4dfa2557aeaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:32.521114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cb172-2c2b-4d87-a23d-4dfa2557aeaa service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:32.542601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 827/1655] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:32.543459 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cb172-2c2b-4d87-a23d-4dfa2557aeaa service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:32.546606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 829/1656] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:56:32.558627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c07cb172-2c2b-4d87-a23d-4dfa2557aeaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:32.698825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cb172-2c2b-4d87-a23d-4dfa2557aeaa service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:32.703369 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25c211-cfcf-489b-8878-53d990108262 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:32.704231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25c211-cfcf-489b-8878-53d990108262 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:32.704643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25c211-cfcf-489b-8878-53d990108262 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:32.705047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25c211-cfcf-489b-8878-53d990108262 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:32.719726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25c211-cfcf-489b-8878-53d990108262 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:32.721608 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25c211-cfcf-489b-8878-53d990108262 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:32.741636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 828/1657] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:33.479387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:34.315902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684dfbea-f4fb-4a87-9f63-26815374dbb0 service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:34.317859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-684dfbea-f4fb-4a87-9f63-26815374dbb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:34.344768 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684dfbea-f4fb-4a87-9f63-26815374dbb0 service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:34.369205 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:34.369969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:34.370325 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:34.370886 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182 admin admin] PATH_INFO: `/v3/projects/f1741ba78495437c8b24846da584f507` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:34.372645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1741ba78495437c8b24846da584f507)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:34.374497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: WARNING py.warnings [None req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:56:34.374497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: warnings.warn(msg) Aug 30 14:56:34.374497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: Aug 30 14:56:34.376483 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:34.596918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1741ba78495437c8b24846da584f507', 'request_id': 'req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:34.605213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1741ba78495437c8b24846da584f507', 'request_id': 'req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:34.679916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1741ba78495437c8b24846da584f507', 'request_id': 'req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:34.724252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182 admin admin] The token cache is being invalidate because project f1741ba78495437c8b24846da584f507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:34.724593 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:34.727284 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:34.734245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 830/1658] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:34 2022] DELETE /identity/v3/projects/f1741ba78495437c8b24846da584f507 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:34.735453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25c211-cfcf-489b-8878-53d990108262 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:34.750349 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25c211-cfcf-489b-8878-53d990108262 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:34.844859 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754694a5-c2d1-4efc-a354-db46ff807b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:34.845564 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754694a5-c2d1-4efc-a354-db46ff807b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:34.845846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754694a5-c2d1-4efc-a354-db46ff807b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:34.846262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754694a5-c2d1-4efc-a354-db46ff807b0d admin admin] PATH_INFO: `/v3/users/cbe21d7e447949668d1081c7e1ae0dfe` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:34.848832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754694a5-c2d1-4efc-a354-db46ff807b0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbe21d7e447949668d1081c7e1ae0dfe)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:34.850198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754694a5-c2d1-4efc-a354-db46ff807b0d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:35.022813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:35.050322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:35.051491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:35.090862 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad87cb8-3b8e-45a7-87bb-28831fcb4182 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:35.110528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-754694a5-c2d1-4efc-a354-db46ff807b0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbe21d7e447949668d1081c7e1ae0dfe'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:35.117710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc538cb-4c19-42d5-a467-e6755d1bf85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:35.120727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc538cb-4c19-42d5-a467-e6755d1bf85c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:35.121014 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc538cb-4c19-42d5-a467-e6755d1bf85c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:35.121275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc538cb-4c19-42d5-a467-e6755d1bf85c admin admin] PATH_INFO: `/v3/projects/fc121f6c44fb497399c6a7f34653066c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:35.125488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc538cb-4c19-42d5-a467-e6755d1bf85c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc121f6c44fb497399c6a7f34653066c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:35.153920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc538cb-4c19-42d5-a467-e6755d1bf85c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:35.154278 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-754694a5-c2d1-4efc-a354-db46ff807b0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbe21d7e447949668d1081c7e1ae0dfe'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:35.157176 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-754694a5-c2d1-4efc-a354-db46ff807b0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbe21d7e447949668d1081c7e1ae0dfe'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:35.248041 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-754694a5-c2d1-4efc-a354-db46ff807b0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbe21d7e447949668d1081c7e1ae0dfe'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:35.290688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-754694a5-c2d1-4efc-a354-db46ff807b0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbe21d7e447949668d1081c7e1ae0dfe'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:35.307826 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 829/1659] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:34 2022] DELETE /identity/v3/users/cbe21d7e447949668d1081c7e1ae0dfe => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:35.377774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cbc538cb-4c19-42d5-a467-e6755d1bf85c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc121f6c44fb497399c6a7f34653066c', 'request_id': 'req-cbc538cb-4c19-42d5-a467-e6755d1bf85c'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:35.403736 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cbc538cb-4c19-42d5-a467-e6755d1bf85c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc121f6c44fb497399c6a7f34653066c', 'request_id': 'req-cbc538cb-4c19-42d5-a467-e6755d1bf85c'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:35.475134 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cbc538cb-4c19-42d5-a467-e6755d1bf85c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc121f6c44fb497399c6a7f34653066c', 'request_id': 'req-cbc538cb-4c19-42d5-a467-e6755d1bf85c'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:35.495193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cbc538cb-4c19-42d5-a467-e6755d1bf85c admin admin] The token cache is being invalidate because project fc121f6c44fb497399c6a7f34653066c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:35.496426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cbc538cb-4c19-42d5-a467-e6755d1bf85c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:35.498851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cbc538cb-4c19-42d5-a467-e6755d1bf85c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:35.502311 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 831/1660] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:35 2022] DELETE /identity/v3/projects/fc121f6c44fb497399c6a7f34653066c => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:35.507260 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754694a5-c2d1-4efc-a354-db46ff807b0d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:35.510662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-754694a5-c2d1-4efc-a354-db46ff807b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:35.809346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754694a5-c2d1-4efc-a354-db46ff807b0d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:35.972835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeccc75-bf0c-443d-b4df-2c89bf9d4095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:35.973713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeccc75-bf0c-443d-b4df-2c89bf9d4095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:35.974123 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeccc75-bf0c-443d-b4df-2c89bf9d4095 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:35.974531 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeccc75-bf0c-443d-b4df-2c89bf9d4095 admin admin] PATH_INFO: `/v3/users/463ffe94e0de477f9d5941ed1c375174` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:35.976155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeccc75-bf0c-443d-b4df-2c89bf9d4095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=463ffe94e0de477f9d5941ed1c375174)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:35.978661 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeccc75-bf0c-443d-b4df-2c89bf9d4095 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:36.221411 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-bbeccc75-bf0c-443d-b4df-2c89bf9d4095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '463ffe94e0de477f9d5941ed1c375174'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:36.253610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-bbeccc75-bf0c-443d-b4df-2c89bf9d4095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '463ffe94e0de477f9d5941ed1c375174'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:36.255350 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-bbeccc75-bf0c-443d-b4df-2c89bf9d4095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '463ffe94e0de477f9d5941ed1c375174'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:36.371380 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-bbeccc75-bf0c-443d-b4df-2c89bf9d4095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '463ffe94e0de477f9d5941ed1c375174'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:36.412384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-bbeccc75-bf0c-443d-b4df-2c89bf9d4095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '463ffe94e0de477f9d5941ed1c375174'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:36.422958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 830/1661] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:35 2022] DELETE /identity/v3/users/463ffe94e0de477f9d5941ed1c375174 => generated 0 bytes in 914 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:36.703455 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc538cb-4c19-42d5-a467-e6755d1bf85c admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:36.705491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-cbc538cb-4c19-42d5-a467-e6755d1bf85c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:36.733498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc538cb-4c19-42d5-a467-e6755d1bf85c admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:36.734308 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:36.878910 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd46ef-4865-4ac0-95d8-f905575d3c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:36.879610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd46ef-4865-4ac0-95d8-f905575d3c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:36.879897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd46ef-4865-4ac0-95d8-f905575d3c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:36.880177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd46ef-4865-4ac0-95d8-f905575d3c26 admin admin] PATH_INFO: `/v3/projects/8df76135f0a84ffbaf2887686f084463` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:36.885500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd46ef-4865-4ac0-95d8-f905575d3c26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8df76135f0a84ffbaf2887686f084463)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:36.893533 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd46ef-4865-4ac0-95d8-f905575d3c26 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:36.906459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:37.119682 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-76dd46ef-4865-4ac0-95d8-f905575d3c26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8df76135f0a84ffbaf2887686f084463', 'request_id': 'req-76dd46ef-4865-4ac0-95d8-f905575d3c26'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:37.131487 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-76dd46ef-4865-4ac0-95d8-f905575d3c26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8df76135f0a84ffbaf2887686f084463', 'request_id': 'req-76dd46ef-4865-4ac0-95d8-f905575d3c26'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:37.245356 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-76dd46ef-4865-4ac0-95d8-f905575d3c26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8df76135f0a84ffbaf2887686f084463', 'request_id': 'req-76dd46ef-4865-4ac0-95d8-f905575d3c26'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:37.269818 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-76dd46ef-4865-4ac0-95d8-f905575d3c26 admin admin] The token cache is being invalidate because project 8df76135f0a84ffbaf2887686f084463 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:37.271952 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-76dd46ef-4865-4ac0-95d8-f905575d3c26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:37.272888 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-76dd46ef-4865-4ac0-95d8-f905575d3c26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:37.276319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 832/1662] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:36 2022] DELETE /identity/v3/projects/8df76135f0a84ffbaf2887686f084463 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:37.281386 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb45963-4659-43d8-9e8f-bd773fd416fd None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:37.288011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb45963-4659-43d8-9e8f-bd773fd416fd None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:37.288011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb45963-4659-43d8-9e8f-bd773fd416fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:37.321054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ebb45963-4659-43d8-9e8f-bd773fd416fd None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:37.323005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ebb45963-4659-43d8-9e8f-bd773fd416fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:37.434184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 831/1663] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:37.442840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9e19-1fb2-4141-bd1e-0ab37e330c89 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:37.444029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9e19-1fb2-4141-bd1e-0ab37e330c89 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:37.444029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9e19-1fb2-4141-bd1e-0ab37e330c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:37.489472 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-77bf9e19-1fb2-4141-bd1e-0ab37e330c89 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:37.491683 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-77bf9e19-1fb2-4141-bd1e-0ab37e330c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:37.505568 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb45963-4659-43d8-9e8f-bd773fd416fd None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:37.514564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 833/1664] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:37.518840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb45963-4659-43d8-9e8f-bd773fd416fd None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:37.524335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e486a8-0722-4bf1-9116-ad866fa970b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:37.525089 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e486a8-0722-4bf1-9116-ad866fa970b6 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:37.525483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e486a8-0722-4bf1-9116-ad866fa970b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:37.526208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e486a8-0722-4bf1-9116-ad866fa970b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:37.526827 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e486a8-0722-4bf1-9116-ad866fa970b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:37.527327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e486a8-0722-4bf1-9116-ad866fa970b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:37.529590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e486a8-0722-4bf1-9116-ad866fa970b6 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:37.543159 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 832/1665] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:37.549283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9e19-1fb2-4141-bd1e-0ab37e330c89 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:37.575964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9e19-1fb2-4141-bd1e-0ab37e330c89 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:37.606750 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f35067-da8b-46ff-a5ed-569e8956d93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:37.610511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f35067-da8b-46ff-a5ed-569e8956d93f admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:37.611893 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f35067-da8b-46ff-a5ed-569e8956d93f admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:37.613832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f35067-da8b-46ff-a5ed-569e8956d93f admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:37.615599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f35067-da8b-46ff-a5ed-569e8956d93f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:37.621236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f35067-da8b-46ff-a5ed-569e8956d93f admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:37.764253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 834/1666] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:37 2022] POST /identity/v3/projects => generated 379 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:37.767875 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e486a8-0722-4bf1-9116-ad866fa970b6 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:37.788759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e486a8-0722-4bf1-9116-ad866fa970b6 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:37.946778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327801f-48ea-4dab-8abe-361d58e9dfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:37.946778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327801f-48ea-4dab-8abe-361d58e9dfbc admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:37.946778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327801f-48ea-4dab-8abe-361d58e9dfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:37.946778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327801f-48ea-4dab-8abe-361d58e9dfbc admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:37.947054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327801f-48ea-4dab-8abe-361d58e9dfbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:37.947054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327801f-48ea-4dab-8abe-361d58e9dfbc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:37.979355 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f35067-da8b-46ff-a5ed-569e8956d93f admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:37.987452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c1f35067-da8b-46ff-a5ed-569e8956d93f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:38.001130 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f35067-da8b-46ff-a5ed-569e8956d93f admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:38.006547 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047026d-7428-4c46-a9aa-5928c0c5816f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:38.007247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047026d-7428-4c46-a9aa-5928c0c5816f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:38.007529 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047026d-7428-4c46-a9aa-5928c0c5816f admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:38.007803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047026d-7428-4c46-a9aa-5928c0c5816f admin admin] PATH_INFO: `/v3/projects/2b71357a6c004abe891759173d465629` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:38.008983 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047026d-7428-4c46-a9aa-5928c0c5816f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b71357a6c004abe891759173d465629)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:38.011291 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047026d-7428-4c46-a9aa-5928c0c5816f admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:38.159059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 833/1667] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:37 2022] POST /identity/v3/users => generated 290 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:38.159849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30f3c3-337c-4975-8f04-2c7c5bc9d2c7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:38.160107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30f3c3-337c-4975-8f04-2c7c5bc9d2c7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:38.160384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30f3c3-337c-4975-8f04-2c7c5bc9d2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:38.211203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e047026d-7428-4c46-a9aa-5928c0c5816f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b71357a6c004abe891759173d465629', 'request_id': 'req-e047026d-7428-4c46-a9aa-5928c0c5816f'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:38.239251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e047026d-7428-4c46-a9aa-5928c0c5816f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b71357a6c004abe891759173d465629', 'request_id': 'req-e047026d-7428-4c46-a9aa-5928c0c5816f'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:38.284163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0b30f3c3-337c-4975-8f04-2c7c5bc9d2c7 None None] MFA Rules not processed for user `07c1825673d7451ba7eb551fc1983228`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:38.296464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0b30f3c3-337c-4975-8f04-2c7c5bc9d2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:38.318534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e047026d-7428-4c46-a9aa-5928c0c5816f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b71357a6c004abe891759173d465629', 'request_id': 'req-e047026d-7428-4c46-a9aa-5928c0c5816f'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:38.337897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e047026d-7428-4c46-a9aa-5928c0c5816f admin admin] The token cache is being invalidate because project 2b71357a6c004abe891759173d465629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:38.338489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e047026d-7428-4c46-a9aa-5928c0c5816f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:38.340894 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e047026d-7428-4c46-a9aa-5928c0c5816f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:38.347900 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 835/1668] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:37 2022] DELETE /identity/v3/projects/2b71357a6c004abe891759173d465629 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:38.350593 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047026d-7428-4c46-a9aa-5928c0c5816f admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:38.355519 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e047026d-7428-4c46-a9aa-5928c0c5816f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:38.507629 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047026d-7428-4c46-a9aa-5928c0c5816f admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:38.519017 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094daf5c-8976-4b90-a6d6-9d4dea394e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:38.519726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094daf5c-8976-4b90-a6d6-9d4dea394e36 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:38.520005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094daf5c-8976-4b90-a6d6-9d4dea394e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:38.520282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094daf5c-8976-4b90-a6d6-9d4dea394e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:38.521201 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094daf5c-8976-4b90-a6d6-9d4dea394e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:38.523366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094daf5c-8976-4b90-a6d6-9d4dea394e36 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:38.544247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 836/1669] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:38.544247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d79b1c-0753-49ea-a629-590f1d898025 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:38.544247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d79b1c-0753-49ea-a629-590f1d898025 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:38.544247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d79b1c-0753-49ea-a629-590f1d898025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:38.557065 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 834/1670] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:38.557065 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30f3c3-337c-4975-8f04-2c7c5bc9d2c7 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:38.597689 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-10d79b1c-0753-49ea-a629-590f1d898025 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:38.598799 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-10d79b1c-0753-49ea-a629-590f1d898025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:38.602162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30f3c3-337c-4975-8f04-2c7c5bc9d2c7 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:38.618979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a62c3-5c30-4ebc-a44f-d613b8968232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:38.619778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a62c3-5c30-4ebc-a44f-d613b8968232 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:38.620062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a62c3-5c30-4ebc-a44f-d613b8968232 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:38.621118 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a62c3-5c30-4ebc-a44f-d613b8968232 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:38.622552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 837/1671] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:38.624017 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3bb6d-460e-417f-b0b0-2ea4194ec2ed None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:38.624304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3bb6d-460e-417f-b0b0-2ea4194ec2ed None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:38.624580 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3bb6d-460e-417f-b0b0-2ea4194ec2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:38.644350 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a62c3-5c30-4ebc-a44f-d613b8968232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:38.646031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a62c3-5c30-4ebc-a44f-d613b8968232 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:38.686046 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-abb3bb6d-460e-417f-b0b0-2ea4194ec2ed None None] MFA Rules not processed for user `07c1825673d7451ba7eb551fc1983228`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:38.687223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-abb3bb6d-460e-417f-b0b0-2ea4194ec2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:38.695051 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 835/1672] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:38.696406 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b0521-0bf1-4db9-8a03-4efcbe43415d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:38.696803 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b0521-0bf1-4db9-8a03-4efcbe43415d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:38.696972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b0521-0bf1-4db9-8a03-4efcbe43415d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:38.718936 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 838/1673] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:38.719578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3bb6d-460e-417f-b0b0-2ea4194ec2ed None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:38.732659 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:38.742943 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3bb6d-460e-417f-b0b0-2ea4194ec2ed None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:38.756388 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a44551-6198-4884-a8b3-de0857fcc06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:38.757872 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a44551-6198-4884-a8b3-de0857fcc06d admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:38.759273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a44551-6198-4884-a8b3-de0857fcc06d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:38.761133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a44551-6198-4884-a8b3-de0857fcc06d admin admin] PATH_INFO: `/v3/projects/e00a0929118f464195dd33179049dc09/users/a737f87f60fd4b59b252c068467270ca/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:38.775405 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-169b0521-0bf1-4db9-8a03-4efcbe43415d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:38.776798 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-169b0521-0bf1-4db9-8a03-4efcbe43415d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:38.790652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 836/1674] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:38.794040 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b0521-0bf1-4db9-8a03-4efcbe43415d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:38.796095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-169b0521-0bf1-4db9-8a03-4efcbe43415d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:38.797050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:38.834588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a44551-6198-4884-a8b3-de0857fcc06d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e00a0929118f464195dd33179049dc09, user_id=a737f87f60fd4b59b252c068467270ca, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:38.838056 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a44551-6198-4884-a8b3-de0857fcc06d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:38.946826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 839/1675] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:38 2022] PUT /identity/v3/projects/e00a0929118f464195dd33179049dc09/users/a737f87f60fd4b59b252c068467270ca/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:38.947738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a44551-6198-4884-a8b3-de0857fcc06d admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:39.024299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b0521-0bf1-4db9-8a03-4efcbe43415d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:39.147873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a44551-6198-4884-a8b3-de0857fcc06d admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:39.163974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c307b-97df-44e6-a475-1e457b70da9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:39.165150 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c307b-97df-44e6-a475-1e457b70da9c admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:39.166377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c307b-97df-44e6-a475-1e457b70da9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:39.168151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c307b-97df-44e6-a475-1e457b70da9c admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:39.168839 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c307b-97df-44e6-a475-1e457b70da9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:39.169147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c307b-97df-44e6-a475-1e457b70da9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:39.175748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c307b-97df-44e6-a475-1e457b70da9c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:39.209346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 840/1676] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:39.213026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba81a5-2d6b-4a58-9eee-c627f95e3559 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:39.221446 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba81a5-2d6b-4a58-9eee-c627f95e3559 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:39.221865 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba81a5-2d6b-4a58-9eee-c627f95e3559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:39.223393 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b65356-74b2-4896-ba63-cec4aec7fc5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:39.224080 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b65356-74b2-4896-ba63-cec4aec7fc5b service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:39.224994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b65356-74b2-4896-ba63-cec4aec7fc5b service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:39.225761 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b65356-74b2-4896-ba63-cec4aec7fc5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:39.309827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-63ba81a5-2d6b-4a58-9eee-c627f95e3559 None None] MFA Rules not processed for user `a737f87f60fd4b59b252c068467270ca`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:39.314751 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-63ba81a5-2d6b-4a58-9eee-c627f95e3559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:39.325021 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b65356-74b2-4896-ba63-cec4aec7fc5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:39.327498 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b65356-74b2-4896-ba63-cec4aec7fc5b service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:39.355740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 837/1677] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:39.356314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b65356-74b2-4896-ba63-cec4aec7fc5b service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:39.393923 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b65356-74b2-4896-ba63-cec4aec7fc5b service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:39.408378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c8d47-8ea3-474c-a9be-6ef1ba426ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:39.409015 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c8d47-8ea3-474c-a9be-6ef1ba426ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:39.409300 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c8d47-8ea3-474c-a9be-6ef1ba426ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:39.409577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c8d47-8ea3-474c-a9be-6ef1ba426ca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:39.413299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c8d47-8ea3-474c-a9be-6ef1ba426ca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:39.415373 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c8d47-8ea3-474c-a9be-6ef1ba426ca4 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:39.536410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 841/1678] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:39.541261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba81a5-2d6b-4a58-9eee-c627f95e3559 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:39.561675 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba81a5-2d6b-4a58-9eee-c627f95e3559 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:39.565613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 838/1679] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:39 2022] POST /identity/v3/projects => generated 377 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:39.567748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636caa5c-b2a2-4b6c-9216-4a38f3bc2542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:39.568626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636caa5c-b2a2-4b6c-9216-4a38f3bc2542 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:39.569128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636caa5c-b2a2-4b6c-9216-4a38f3bc2542 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:39.569424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636caa5c-b2a2-4b6c-9216-4a38f3bc2542 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:39.570233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636caa5c-b2a2-4b6c-9216-4a38f3bc2542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:39.570781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c8d47-8ea3-474c-a9be-6ef1ba426ca4 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:39.576439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636caa5c-b2a2-4b6c-9216-4a38f3bc2542 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:39.704390 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c8d47-8ea3-474c-a9be-6ef1ba426ca4 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:39.722441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 842/1680] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:39 2022] POST /identity/v3/projects => generated 379 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:39.726279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636caa5c-b2a2-4b6c-9216-4a38f3bc2542 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:39.751781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f1a1f-b8ed-4125-a2f4-f9f2d06c3f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:39.752425 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f1a1f-b8ed-4125-a2f4-f9f2d06c3f44 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:39.756894 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f1a1f-b8ed-4125-a2f4-f9f2d06c3f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:39.757265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f1a1f-b8ed-4125-a2f4-f9f2d06c3f44 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:39.758477 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f1a1f-b8ed-4125-a2f4-f9f2d06c3f44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:39.763292 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f1a1f-b8ed-4125-a2f4-f9f2d06c3f44 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:39.881908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636caa5c-b2a2-4b6c-9216-4a38f3bc2542 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:39.891579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 839/1681] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:39 2022] POST /identity/v3/users => generated 289 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:56:39.895692 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f1a1f-b8ed-4125-a2f4-f9f2d06c3f44 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:39.900611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b98df5-7180-42ac-abe8-f29577f5f207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:39.901245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98df5-7180-42ac-abe8-f29577f5f207 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:39.904271 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98df5-7180-42ac-abe8-f29577f5f207 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:39.904271 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98df5-7180-42ac-abe8-f29577f5f207 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:39.904271 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b98df5-7180-42ac-abe8-f29577f5f207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:39.906814 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b98df5-7180-42ac-abe8-f29577f5f207 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:39.923047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f1a1f-b8ed-4125-a2f4-f9f2d06c3f44 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:39.940252 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c27d2e-6fc1-4361-9b5e-1b9df4d0ab72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:39.941657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c27d2e-6fc1-4361-9b5e-1b9df4d0ab72 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:39.942175 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c27d2e-6fc1-4361-9b5e-1b9df4d0ab72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:39.944299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c27d2e-6fc1-4361-9b5e-1b9df4d0ab72 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:39.947599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c27d2e-6fc1-4361-9b5e-1b9df4d0ab72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:39.949577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c27d2e-6fc1-4361-9b5e-1b9df4d0ab72 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:39.967150 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 840/1682] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:56:39.972390 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c27d2e-6fc1-4361-9b5e-1b9df4d0ab72 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:39.984313 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c27d2e-6fc1-4361-9b5e-1b9df4d0ab72 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:39.990649 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c37a5-4f7e-4a30-93c9-a08e8161a6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:39.991766 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c37a5-4f7e-4a30-93c9-a08e8161a6e3 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:39.992249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c37a5-4f7e-4a30-93c9-a08e8161a6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:39.992719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c37a5-4f7e-4a30-93c9-a08e8161a6e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:39.993581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c37a5-4f7e-4a30-93c9-a08e8161a6e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:39.996125 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c37a5-4f7e-4a30-93c9-a08e8161a6e3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:40.012362 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 841/1683] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:56:40.017976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c37a5-4f7e-4a30-93c9-a08e8161a6e3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:40.040789 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c37a5-4f7e-4a30-93c9-a08e8161a6e3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:40.058184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f259a09-80d2-4d39-990d-d42695e63a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:40.059936 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f259a09-80d2-4d39-990d-d42695e63a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:40.060293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f259a09-80d2-4d39-990d-d42695e63a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:40.060750 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f259a09-80d2-4d39-990d-d42695e63a2a admin admin] PATH_INFO: `/v3/projects/05b7c5a9fa3f4c41bb4700bf58f26d0f/users/800a15ff9e154cf3ae89dc7a4282f247/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:40.068948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 843/1684] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:39 2022] POST /identity/v3/users => generated 296 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:56:40.073095 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b98df5-7180-42ac-abe8-f29577f5f207 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:40.109094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f259a09-80d2-4d39-990d-d42695e63a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b7c5a9fa3f4c41bb4700bf58f26d0f, user_id=800a15ff9e154cf3ae89dc7a4282f247, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:40.116823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b98df5-7180-42ac-abe8-f29577f5f207 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:40.126715 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f259a09-80d2-4d39-990d-d42695e63a2a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:40.127795 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccb349-5cf9-43a5-b850-bb5a9813e700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:40.128673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccb349-5cf9-43a5-b850-bb5a9813e700 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:40.134026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccb349-5cf9-43a5-b850-bb5a9813e700 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:40.134026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccb349-5cf9-43a5-b850-bb5a9813e700 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:40.134026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ccb349-5cf9-43a5-b850-bb5a9813e700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:40.134026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ccb349-5cf9-43a5-b850-bb5a9813e700 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:40.164843 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 844/1685] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:40.169076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccb349-5cf9-43a5-b850-bb5a9813e700 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:40.198811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccb349-5cf9-43a5-b850-bb5a9813e700 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:40.232453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 842/1686] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:40 2022] PUT /identity/v3/projects/05b7c5a9fa3f4c41bb4700bf58f26d0f/users/800a15ff9e154cf3ae89dc7a4282f247/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:40.232957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce185154-7eac-4f6e-a992-40bf3556cfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:40.232957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce185154-7eac-4f6e-a992-40bf3556cfbe admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:40.232957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce185154-7eac-4f6e-a992-40bf3556cfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:40.232957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce185154-7eac-4f6e-a992-40bf3556cfbe admin admin] PATH_INFO: `/v3/projects/8739a250ea1f400eb3fb88c72c312ac4/users/e9bb53cc6bf74405a3d29d5fc11a21ac/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:40.235048 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dadbec-4bf1-446e-9d5a-fb42b02ef7f1 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:40.235490 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dadbec-4bf1-446e-9d5a-fb42b02ef7f1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:40.235884 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dadbec-4bf1-446e-9d5a-fb42b02ef7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:40.302742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce185154-7eac-4f6e-a992-40bf3556cfbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8739a250ea1f400eb3fb88c72c312ac4, user_id=e9bb53cc6bf74405a3d29d5fc11a21ac, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:40.305650 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-61dadbec-4bf1-446e-9d5a-fb42b02ef7f1 None None] MFA Rules not processed for user `800a15ff9e154cf3ae89dc7a4282f247`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:40.306803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce185154-7eac-4f6e-a992-40bf3556cfbe admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:40.306915 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-61dadbec-4bf1-446e-9d5a-fb42b02ef7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:40.424316 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 845/1687] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:40 2022] PUT /identity/v3/projects/8739a250ea1f400eb3fb88c72c312ac4/users/e9bb53cc6bf74405a3d29d5fc11a21ac/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:40.431148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce185154-7eac-4f6e-a992-40bf3556cfbe admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:40.469089 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 843/1688] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:40.481253 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dadbec-4bf1-446e-9d5a-fb42b02ef7f1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:40.482706 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-61dadbec-4bf1-446e-9d5a-fb42b02ef7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:40.586081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:40.622825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce185154-7eac-4f6e-a992-40bf3556cfbe admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:40.628859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488d09a-2daa-46b2-ad7f-a6385bbaddc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:40.629559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488d09a-2daa-46b2-ad7f-a6385bbaddc9 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:40.629966 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488d09a-2daa-46b2-ad7f-a6385bbaddc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:40.630388 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488d09a-2daa-46b2-ad7f-a6385bbaddc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:40.632588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488d09a-2daa-46b2-ad7f-a6385bbaddc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:40.634707 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488d09a-2daa-46b2-ad7f-a6385bbaddc9 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:40.653191 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 846/1689] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:40.659738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd6ac1-134d-402c-9aaa-16711cc980a9 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:40.660576 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd6ac1-134d-402c-9aaa-16711cc980a9 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:40.660867 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd6ac1-134d-402c-9aaa-16711cc980a9 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:40.675813 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 847/1690] 10.222.0.27 () {54 vars in 912 bytes} [Tue Aug 30 14:56:40 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:56:40.676753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd6ac1-134d-402c-9aaa-16711cc980a9 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:40.704657 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd6ac1-134d-402c-9aaa-16711cc980a9 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:40.719832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc60e5-e1c1-4117-8c6c-7804b4ba2669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:40.723246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc60e5-e1c1-4117-8c6c-7804b4ba2669 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:40.723676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc60e5-e1c1-4117-8c6c-7804b4ba2669 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:40.727228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc60e5-e1c1-4117-8c6c-7804b4ba2669 admin admin] PATH_INFO: `/v3/projects/8739a250ea1f400eb3fb88c72c312ac4/users/e9bb53cc6bf74405a3d29d5fc11a21ac/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:40.729412 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc60e5-e1c1-4117-8c6c-7804b4ba2669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8739a250ea1f400eb3fb88c72c312ac4, user_id=e9bb53cc6bf74405a3d29d5fc11a21ac, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:40.733660 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc60e5-e1c1-4117-8c6c-7804b4ba2669 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:40.751316 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:40.753998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 848/1691] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:40 2022] PUT /identity/v3/projects/8739a250ea1f400eb3fb88c72c312ac4/users/e9bb53cc6bf74405a3d29d5fc11a21ac/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:40.756988 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c20aa-da7b-4359-8a9c-ef36342f0003 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:40.757595 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c20aa-da7b-4359-8a9c-ef36342f0003 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:40.757966 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c20aa-da7b-4359-8a9c-ef36342f0003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:40.769024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dadbec-4bf1-446e-9d5a-fb42b02ef7f1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:40.881316 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-ed7c20aa-da7b-4359-8a9c-ef36342f0003 None None] MFA Rules not processed for user `7bff0664e0b44b7c85344a9f3e88f126`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:40.887659 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ed7c20aa-da7b-4359-8a9c-ef36342f0003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:40.894361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2386e8a-a559-4b37-90f9-005bb4b9b1f4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfda7307e703427faaf599e32b91bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:40.894988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2386e8a-a559-4b37-90f9-005bb4b9b1f4 service glance] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:40.895297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2386e8a-a559-4b37-90f9-005bb4b9b1f4 service glance] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:40.895573 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2386e8a-a559-4b37-90f9-005bb4b9b1f4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:40.983561 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 849/1692] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 30 14:56:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:40.986010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c20aa-da7b-4359-8a9c-ef36342f0003 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:40.987684 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ed7c20aa-da7b-4359-8a9c-ef36342f0003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:41.016021 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2386e8a-a559-4b37-90f9-005bb4b9b1f4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:41.019816 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2386e8a-a559-4b37-90f9-005bb4b9b1f4 service glance] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:41.047942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 844/1693] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:41.050059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a816a4e-1874-4c63-b823-a44ee0fee050 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:41.051454 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a816a4e-1874-4c63-b823-a44ee0fee050 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:41.051739 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a816a4e-1874-4c63-b823-a44ee0fee050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:41.139892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0a816a4e-1874-4c63-b823-a44ee0fee050 None None] MFA Rules not processed for user `e9bb53cc6bf74405a3d29d5fc11a21ac`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:41.139892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0a816a4e-1874-4c63-b823-a44ee0fee050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:41.160006 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:41.179691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c20aa-da7b-4359-8a9c-ef36342f0003 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:41.194007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a13e1-6c38-4743-afdd-e2a56d344b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:41.194719 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a13e1-6c38-4743-afdd-e2a56d344b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:41.195002 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a13e1-6c38-4743-afdd-e2a56d344b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:41.195295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a13e1-6c38-4743-afdd-e2a56d344b88 admin admin] PATH_INFO: `/v3/users/3f3e15e7a6354ca99e8635a0948af26d` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:41.198940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a13e1-6c38-4743-afdd-e2a56d344b88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f3e15e7a6354ca99e8635a0948af26d)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:41.203425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a13e1-6c38-4743-afdd-e2a56d344b88 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:41.287255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 845/1694] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:41.289624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dac59c-4072-4fc4-8d9f-b637ec158735 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:41.289842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dac59c-4072-4fc4-8d9f-b637ec158735 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:41.290128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dac59c-4072-4fc4-8d9f-b637ec158735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:41.330310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b5dac59c-4072-4fc4-8d9f-b637ec158735 None None] MFA Rules not processed for user `7dc5cd44b1b34a7ebbdca2c062715431`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:41.330310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b5dac59c-4072-4fc4-8d9f-b637ec158735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:41.409753 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 846/1695] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:41 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:41.410580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dac59c-4072-4fc4-8d9f-b637ec158735 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:41.412951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b5dac59c-4072-4fc4-8d9f-b637ec158735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:41.426230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dac59c-4072-4fc4-8d9f-b637ec158735 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:41.430682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3307950-8504-40a3-b4f8-da18f2c9ab10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:41.433649 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3307950-8504-40a3-b4f8-da18f2c9ab10 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:41.433649 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3307950-8504-40a3-b4f8-da18f2c9ab10 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:41.433649 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3307950-8504-40a3-b4f8-da18f2c9ab10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:41.444215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3307950-8504-40a3-b4f8-da18f2c9ab10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:41.448655 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3307950-8504-40a3-b4f8-da18f2c9ab10 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:41.484987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 847/1696] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:41.489315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3307950-8504-40a3-b4f8-da18f2c9ab10 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:41.502535 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f3307950-8504-40a3-b4f8-da18f2c9ab10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:41.504742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-385a13e1-6c38-4743-afdd-e2a56d344b88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f3e15e7a6354ca99e8635a0948af26d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:41.521688 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-385a13e1-6c38-4743-afdd-e2a56d344b88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f3e15e7a6354ca99e8635a0948af26d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:41.553198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-385a13e1-6c38-4743-afdd-e2a56d344b88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f3e15e7a6354ca99e8635a0948af26d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:41.553876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-385a13e1-6c38-4743-afdd-e2a56d344b88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f3e15e7a6354ca99e8635a0948af26d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:41.554498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-385a13e1-6c38-4743-afdd-e2a56d344b88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f3e15e7a6354ca99e8635a0948af26d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:41.636635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3307950-8504-40a3-b4f8-da18f2c9ab10 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:41.653606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d7f55-dd9f-4aac-885d-ef075a761758 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:41.654247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d7f55-dd9f-4aac-885d-ef075a761758 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:41.654523 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d7f55-dd9f-4aac-885d-ef075a761758 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:41.654792 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d7f55-dd9f-4aac-885d-ef075a761758 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:41.656650 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-231d7f55-dd9f-4aac-885d-ef075a761758 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:41.665923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 850/1697] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:40 2022] DELETE /identity/v3/users/3f3e15e7a6354ca99e8635a0948af26d => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:41.669638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a13e1-6c38-4743-afdd-e2a56d344b88 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:41.671056 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-385a13e1-6c38-4743-afdd-e2a56d344b88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:41.713583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d7f55-dd9f-4aac-885d-ef075a761758 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:41.716549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d7f55-dd9f-4aac-885d-ef075a761758 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:41.753099 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 848/1698] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:41.759572 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b627b7-1d60-4b34-ad5f-c0a92c149561 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:41.759887 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b627b7-1d60-4b34-ad5f-c0a92c149561 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:41.760121 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b627b7-1d60-4b34-ad5f-c0a92c149561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:41.830825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-97b627b7-1d60-4b34-ad5f-c0a92c149561 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:41.832097 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-97b627b7-1d60-4b34-ad5f-c0a92c149561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:41.923558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a13e1-6c38-4743-afdd-e2a56d344b88 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:41.933247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfbda96-8136-4c7f-958a-83eb5e8518ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:41.933584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbda96-8136-4c7f-958a-83eb5e8518ea service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:41.935403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbda96-8136-4c7f-958a-83eb5e8518ea service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:41.935691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbda96-8136-4c7f-958a-83eb5e8518ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:41.938349 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1bfbda96-8136-4c7f-958a-83eb5e8518ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:42.006593 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:42.080835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfbda96-8136-4c7f-958a-83eb5e8518ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:42.094455 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfbda96-8136-4c7f-958a-83eb5e8518ea service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:42.110582 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 849/1699] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:42.117737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b627b7-1d60-4b34-ad5f-c0a92c149561 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:42.119473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-97b627b7-1d60-4b34-ad5f-c0a92c149561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:42.133593 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 851/1700] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:41 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:42.278518 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfbda96-8136-4c7f-958a-83eb5e8518ea service nova] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:42.280369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1bfbda96-8136-4c7f-958a-83eb5e8518ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:42.314947 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b627b7-1d60-4b34-ad5f-c0a92c149561 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:42.344093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccceb74-6e1b-4954-81a8-250a5c2d27af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:42.350973 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccceb74-6e1b-4954-81a8-250a5c2d27af service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:42.351299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccceb74-6e1b-4954-81a8-250a5c2d27af service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:42.353673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccceb74-6e1b-4954-81a8-250a5c2d27af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:42.384262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccceb74-6e1b-4954-81a8-250a5c2d27af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:42.385785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccceb74-6e1b-4954-81a8-250a5c2d27af service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:42.419352 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 850/1701] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:56:42.541625 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfbda96-8136-4c7f-958a-83eb5e8518ea service nova] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:42.550664 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073eb2c-0bef-414c-8810-108103e625a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:42.552826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073eb2c-0bef-414c-8810-108103e625a6 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:42.553133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073eb2c-0bef-414c-8810-108103e625a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:42.553409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073eb2c-0bef-414c-8810-108103e625a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:42.561068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-5073eb2c-0bef-414c-8810-108103e625a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:42.633396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:42.744560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccceb74-6e1b-4954-81a8-250a5c2d27af service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:42.746501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5ccceb74-6e1b-4954-81a8-250a5c2d27af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:42.829900 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073eb2c-0bef-414c-8810-108103e625a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:42.834626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073eb2c-0bef-414c-8810-108103e625a6 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:42.883011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 852/1702] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:56:42.981993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccceb74-6e1b-4954-81a8-250a5c2d27af service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:43.011592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2ad71-6968-49eb-a640-53f68831c1d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4f11655cd51438e9652869f74ffa975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:43.012286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2ad71-6968-49eb-a640-53f68831c1d6 service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:43.012578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2ad71-6968-49eb-a640-53f68831c1d6 service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:43.012852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2ad71-6968-49eb-a640-53f68831c1d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:43.021686 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-07e2ad71-6968-49eb-a640-53f68831c1d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:43.057104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2ad71-6968-49eb-a640-53f68831c1d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:43.058745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2ad71-6968-49eb-a640-53f68831c1d6 service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:43.103966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 851/1703] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:56:43.357092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:43.465903 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:43.645258 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e21207-103b-4ba6-8b6b-a80e4f4a563c None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:43.649089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e21207-103b-4ba6-8b6b-a80e4f4a563c None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:43.649089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e21207-103b-4ba6-8b6b-a80e4f4a563c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:43.655397 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb20433-c43f-4d6b-ae1e-a6b00cf28d03 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:43.655833 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb20433-c43f-4d6b-ae1e-a6b00cf28d03 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:43.655917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb20433-c43f-4d6b-ae1e-a6b00cf28d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:43.746510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ebb20433-c43f-4d6b-ae1e-a6b00cf28d03 None None] MFA Rules not processed for user `3bd31b8807fc450f8f134417cd535542`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:43.748124 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ebb20433-c43f-4d6b-ae1e-a6b00cf28d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:43.778579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-d6e21207-103b-4ba6-8b6b-a80e4f4a563c None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:43.782124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d6e21207-103b-4ba6-8b6b-a80e4f4a563c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:43.931720 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 852/1704] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:56:43.932930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb20433-c43f-4d6b-ae1e-a6b00cf28d03 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:43.934994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ebb20433-c43f-4d6b-ae1e-a6b00cf28d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:43.947340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:43.953695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb20433-c43f-4d6b-ae1e-a6b00cf28d03 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:43.953895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 853/1705] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 30 14:56:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:43.961165 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec4755-032c-4cd4-a560-d62a8b83bbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:43.962262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec4755-032c-4cd4-a560-d62a8b83bbf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:43.962748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec4755-032c-4cd4-a560-d62a8b83bbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:43.965153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec4755-032c-4cd4-a560-d62a8b83bbf9 admin admin] PATH_INFO: `/v3/projects/92b4ddd948934014bd572c1e9a4800e5` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:43.968313 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e21207-103b-4ba6-8b6b-a80e4f4a563c None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:43.970175 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec4755-032c-4cd4-a560-d62a8b83bbf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92b4ddd948934014bd572c1e9a4800e5)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:43.970397 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d6e21207-103b-4ba6-8b6b-a80e4f4a563c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:43.973671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec4755-032c-4cd4-a560-d62a8b83bbf9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:44.036520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e21207-103b-4ba6-8b6b-a80e4f4a563c None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:44.051078 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7d8dd-a6ca-4f4d-80bc-7a8830edc314 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:44.052770 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7d8dd-a6ca-4f4d-80bc-7a8830edc314 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:44.053190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7d8dd-a6ca-4f4d-80bc-7a8830edc314 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:44.053465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7d8dd-a6ca-4f4d-80bc-7a8830edc314 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:44.064684 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7d8dd-a6ca-4f4d-80bc-7a8830edc314 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:44.066293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7d8dd-a6ca-4f4d-80bc-7a8830edc314 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:44.078562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 854/1706] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:44.196117 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-4dec4755-032c-4cd4-a560-d62a8b83bbf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92b4ddd948934014bd572c1e9a4800e5', 'request_id': 'req-4dec4755-032c-4cd4-a560-d62a8b83bbf9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:44.202581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-4dec4755-032c-4cd4-a560-d62a8b83bbf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92b4ddd948934014bd572c1e9a4800e5', 'request_id': 'req-4dec4755-032c-4cd4-a560-d62a8b83bbf9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:44.209179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-4dec4755-032c-4cd4-a560-d62a8b83bbf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92b4ddd948934014bd572c1e9a4800e5', 'request_id': 'req-4dec4755-032c-4cd4-a560-d62a8b83bbf9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:44.301392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-4dec4755-032c-4cd4-a560-d62a8b83bbf9 admin admin] The token cache is being invalidate because project 92b4ddd948934014bd572c1e9a4800e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:56:44.301736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-4dec4755-032c-4cd4-a560-d62a8b83bbf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:44.303988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-4dec4755-032c-4cd4-a560-d62a8b83bbf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:56:44.307563 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 853/1707] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:56:43 2022] DELETE /identity/v3/projects/92b4ddd948934014bd572c1e9a4800e5 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:44.313483 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcaeefd-af81-4d6e-9a32-733151b5d1bb None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:44.313707 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcaeefd-af81-4d6e-9a32-733151b5d1bb None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:44.314026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcaeefd-af81-4d6e-9a32-733151b5d1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:44.359777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-5bcaeefd-af81-4d6e-9a32-733151b5d1bb None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:44.361294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-5bcaeefd-af81-4d6e-9a32-733151b5d1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:44.411078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec4755-032c-4cd4-a560-d62a8b83bbf9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:44.412580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4dec4755-032c-4cd4-a560-d62a8b83bbf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:44.572771 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 855/1708] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:44.575380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d50b9-7f75-404d-9a7c-03f28b5927f2 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:44.575665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d50b9-7f75-404d-9a7c-03f28b5927f2 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:44.577876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d50b9-7f75-404d-9a7c-03f28b5927f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:44.640902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-358d50b9-7f75-404d-9a7c-03f28b5927f2 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:44.645022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-358d50b9-7f75-404d-9a7c-03f28b5927f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:44.682280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 856/1709] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:44.688256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d50b9-7f75-404d-9a7c-03f28b5927f2 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:44.699427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec4755-032c-4cd4-a560-d62a8b83bbf9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:44.715066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d50b9-7f75-404d-9a7c-03f28b5927f2 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:44.720441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b4a48-bdb3-46cc-a1f8-c30a5379e83f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:44.720441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b4a48-bdb3-46cc-a1f8-c30a5379e83f service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:44.720441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b4a48-bdb3-46cc-a1f8-c30a5379e83f service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:44.720802 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b4a48-bdb3-46cc-a1f8-c30a5379e83f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:44.730517 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d96b4a48-bdb3-46cc-a1f8-c30a5379e83f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:44.734933 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cafd3-33e8-4fe7-b9a6-afe9768aae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:44.736165 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cafd3-33e8-4fe7-b9a6-afe9768aae44 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:44.736444 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cafd3-33e8-4fe7-b9a6-afe9768aae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:44.736716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cafd3-33e8-4fe7-b9a6-afe9768aae44 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:44.737372 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193cafd3-33e8-4fe7-b9a6-afe9768aae44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:56:44.737676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193cafd3-33e8-4fe7-b9a6-afe9768aae44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:44.745791 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193cafd3-33e8-4fe7-b9a6-afe9768aae44 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:44.784336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 857/1710] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:56:44.786057 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cafd3-33e8-4fe7-b9a6-afe9768aae44 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:44.812444 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cafd3-33e8-4fe7-b9a6-afe9768aae44 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:44.825755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611a9f7-8e0a-44f9-8d0d-67cbd2fbb2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:44.826439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611a9f7-8e0a-44f9-8d0d-67cbd2fbb2bb admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:44.826725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611a9f7-8e0a-44f9-8d0d-67cbd2fbb2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:44.826999 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611a9f7-8e0a-44f9-8d0d-67cbd2fbb2bb admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:44.828971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611a9f7-8e0a-44f9-8d0d-67cbd2fbb2bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:44.834115 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611a9f7-8e0a-44f9-8d0d-67cbd2fbb2bb admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:44.893685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b4a48-bdb3-46cc-a1f8-c30a5379e83f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:44.899032 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b4a48-bdb3-46cc-a1f8-c30a5379e83f service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:44.926093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 854/1711] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 14:56:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 14:56:44.928130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697cec8-c170-414d-9277-599be0a032c1 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:44.928420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697cec8-c170-414d-9277-599be0a032c1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:44.931473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697cec8-c170-414d-9277-599be0a032c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:44.950618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 858/1712] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:44 2022] POST /identity/v3/projects => generated 407 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:44.955009 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611a9f7-8e0a-44f9-8d0d-67cbd2fbb2bb admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:44.967174 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:44.995643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6697cec8-c170-414d-9277-599be0a032c1 None None] MFA Rules not processed for user `39d98294ba884280b4971af36a9fbd6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:44.996877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6697cec8-c170-414d-9277-599be0a032c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:45.138492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611a9f7-8e0a-44f9-8d0d-67cbd2fbb2bb admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:45.170496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 855/1713] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:45.179960 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1dc483-8905-41fa-8ba9-880436aa04f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:45.180919 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3fc66-01a4-478d-85d5-da3d0219b44c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:45.181377 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3fc66-01a4-478d-85d5-da3d0219b44c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:45.181819 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3fc66-01a4-478d-85d5-da3d0219b44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:45.182567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1dc483-8905-41fa-8ba9-880436aa04f0 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:45.184809 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1dc483-8905-41fa-8ba9-880436aa04f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:45.186066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1dc483-8905-41fa-8ba9-880436aa04f0 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:45.189988 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1dc483-8905-41fa-8ba9-880436aa04f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:45.199405 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1dc483-8905-41fa-8ba9-880436aa04f0 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:45.216576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-fde3fc66-01a4-478d-85d5-da3d0219b44c None None] MFA Rules not processed for user `39d98294ba884280b4971af36a9fbd6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:45.220228 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fde3fc66-01a4-478d-85d5-da3d0219b44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:45.271465 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 856/1714] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:56:45.297905 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3fc66-01a4-478d-85d5-da3d0219b44c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:45.300064 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fde3fc66-01a4-478d-85d5-da3d0219b44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:45.354311 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 859/1715] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:44 2022] POST /identity/v3/users => generated 304 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:56:45.356101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1dc483-8905-41fa-8ba9-880436aa04f0 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:45.386995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1dc483-8905-41fa-8ba9-880436aa04f0 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:45.404799 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650e837-f536-4f0a-8f66-3c81774f50ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:45.405987 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650e837-f536-4f0a-8f66-3c81774f50ca admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:45.406189 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650e837-f536-4f0a-8f66-3c81774f50ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:45.406604 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650e837-f536-4f0a-8f66-3c81774f50ca admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:45.407604 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650e837-f536-4f0a-8f66-3c81774f50ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:45.409525 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650e837-f536-4f0a-8f66-3c81774f50ca admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:45.434385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 860/1716] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:45.438416 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650e837-f536-4f0a-8f66-3c81774f50ca admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:45.459296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650e837-f536-4f0a-8f66-3c81774f50ca admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:45.480182 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820e81c-996a-46e2-92e0-d9bc05cf9d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:45.481138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820e81c-996a-46e2-92e0-d9bc05cf9d32 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:45.483491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820e81c-996a-46e2-92e0-d9bc05cf9d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:45.484008 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820e81c-996a-46e2-92e0-d9bc05cf9d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:45.485156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820e81c-996a-46e2-92e0-d9bc05cf9d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:45.489320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820e81c-996a-46e2-92e0-d9bc05cf9d32 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:45.521116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 861/1717] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:45.522084 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820e81c-996a-46e2-92e0-d9bc05cf9d32 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:45.546879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820e81c-996a-46e2-92e0-d9bc05cf9d32 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:45.563641 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be6505-52d3-4b58-914d-3c69d3ac5b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:45.564403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be6505-52d3-4b58-914d-3c69d3ac5b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:45.564571 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be6505-52d3-4b58-914d-3c69d3ac5b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:45.564814 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be6505-52d3-4b58-914d-3c69d3ac5b41 admin admin] PATH_INFO: `/v3/projects/773c9f06b9264acab70e57aabcff1b7e/users/a9330ab132a04fe98091567044c9417e/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:45.615824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be6505-52d3-4b58-914d-3c69d3ac5b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=773c9f06b9264acab70e57aabcff1b7e, user_id=a9330ab132a04fe98091567044c9417e, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:45.626493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be6505-52d3-4b58-914d-3c69d3ac5b41 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:45.635264 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3fc66-01a4-478d-85d5-da3d0219b44c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:45.666227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59c250-9946-41e5-88b8-127100a7671e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:45.667508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59c250-9946-41e5-88b8-127100a7671e service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:45.669561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59c250-9946-41e5-88b8-127100a7671e service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:45.671412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59c250-9946-41e5-88b8-127100a7671e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:45.738484 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59c250-9946-41e5-88b8-127100a7671e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:45.739132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 862/1718] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:45 2022] PUT /identity/v3/projects/773c9f06b9264acab70e57aabcff1b7e/users/a9330ab132a04fe98091567044c9417e/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:45.751825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bc9ef-5e9e-410e-827d-f0baf18237f3 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:45.752121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bc9ef-5e9e-410e-827d-f0baf18237f3 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:45.752788 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bc9ef-5e9e-410e-827d-f0baf18237f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:45.758644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59c250-9946-41e5-88b8-127100a7671e service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:45.828058 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-770bc9ef-5e9e-410e-827d-f0baf18237f3 None None] MFA Rules not processed for user `a9330ab132a04fe98091567044c9417e`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:45.829708 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 857/1719] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:45.832923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-770bc9ef-5e9e-410e-827d-f0baf18237f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:46.098967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 863/1720] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:45 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:46.100298 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59c250-9946-41e5-88b8-127100a7671e service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:46.115736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59c250-9946-41e5-88b8-127100a7671e service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:46.201505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0fb25-4d22-4ceb-8d15-dcab036167af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:46.202293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0fb25-4d22-4ceb-8d15-dcab036167af admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:46.202577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0fb25-4d22-4ceb-8d15-dcab036167af admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:46.202849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f0fb25-4d22-4ceb-8d15-dcab036167af admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:46.203538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0fb25-4d22-4ceb-8d15-dcab036167af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:46.216131 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f0fb25-4d22-4ceb-8d15-dcab036167af admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:46.369076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bc9ef-5e9e-410e-827d-f0baf18237f3 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:46.370841 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 858/1721] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:56:46 2022] POST /identity/v3/projects => generated 407 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:56:46.409538 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bc9ef-5e9e-410e-827d-f0baf18237f3 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:46.517090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:46.786965 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07eb7e6-fb06-42f9-b4ba-afd5d7af0281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:46.786965 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07eb7e6-fb06-42f9-b4ba-afd5d7af0281 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:46.786965 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07eb7e6-fb06-42f9-b4ba-afd5d7af0281 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:46.786965 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07eb7e6-fb06-42f9-b4ba-afd5d7af0281 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:46.790213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07eb7e6-fb06-42f9-b4ba-afd5d7af0281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:46.790213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07eb7e6-fb06-42f9-b4ba-afd5d7af0281 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:47.167030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 864/1722] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:56:46 2022] POST /identity/v3/users => generated 310 bytes in 799 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:56:47.173886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0fb25-4d22-4ceb-8d15-dcab036167af admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:47.184771 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f0fb25-4d22-4ceb-8d15-dcab036167af admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:47.217885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f509d3e-dfd9-4350-ad76-cef9b3617cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:47.221304 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f509d3e-dfd9-4350-ad76-cef9b3617cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:47.221676 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f509d3e-dfd9-4350-ad76-cef9b3617cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:47.222191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f509d3e-dfd9-4350-ad76-cef9b3617cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:47.224558 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f509d3e-dfd9-4350-ad76-cef9b3617cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:47.229180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f509d3e-dfd9-4350-ad76-cef9b3617cc5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:47.317616 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 859/1723] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:47.318911 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07eb7e6-fb06-42f9-b4ba-afd5d7af0281 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:47.334944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07eb7e6-fb06-42f9-b4ba-afd5d7af0281 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:47.354865 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb7379-56ee-4091-8a87-00df1b631c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:47.355866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb7379-56ee-4091-8a87-00df1b631c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:47.356229 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb7379-56ee-4091-8a87-00df1b631c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:47.356651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb7379-56ee-4091-8a87-00df1b631c4e admin admin] PATH_INFO: `/v3/projects/27b98bc5cfba49fb8a63b1942b3ec09d/users/d4d9687d9cea45f49fab15eb66ac15ab/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:47.449681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb7379-56ee-4091-8a87-00df1b631c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b98bc5cfba49fb8a63b1942b3ec09d, user_id=d4d9687d9cea45f49fab15eb66ac15ab, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:47.456014 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb7379-56ee-4091-8a87-00df1b631c4e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:47.602131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 865/1724] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:47 2022] PUT /identity/v3/projects/27b98bc5cfba49fb8a63b1942b3ec09d/users/d4d9687d9cea45f49fab15eb66ac15ab/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:47.605781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f509d3e-dfd9-4350-ad76-cef9b3617cc5 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:47.617357 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f509d3e-dfd9-4350-ad76-cef9b3617cc5 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:47.750288 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861dfdc-e866-43fa-951b-bb3b74775cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:47.751018 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861dfdc-e866-43fa-951b-bb3b74775cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:47.751441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861dfdc-e866-43fa-951b-bb3b74775cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:47.752030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861dfdc-e866-43fa-951b-bb3b74775cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:47.754621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861dfdc-e866-43fa-951b-bb3b74775cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:47.756453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861dfdc-e866-43fa-951b-bb3b74775cc6 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:47.801993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 860/1725] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:56:47.803287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb7379-56ee-4091-8a87-00df1b631c4e admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:47.816792 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb7379-56ee-4091-8a87-00df1b631c4e admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:47.834186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df585018-c2a8-4f51-9694-11e9c6be0232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:47.834186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df585018-c2a8-4f51-9694-11e9c6be0232 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:47.834186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df585018-c2a8-4f51-9694-11e9c6be0232 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:47.834186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df585018-c2a8-4f51-9694-11e9c6be0232 admin admin] PATH_INFO: `/v3/projects/27b98bc5cfba49fb8a63b1942b3ec09d/users/d4d9687d9cea45f49fab15eb66ac15ab/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:47.848059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df585018-c2a8-4f51-9694-11e9c6be0232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b98bc5cfba49fb8a63b1942b3ec09d, user_id=d4d9687d9cea45f49fab15eb66ac15ab, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:47.851257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df585018-c2a8-4f51-9694-11e9c6be0232 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:47.888192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28856048-2dde-4bc1-9290-8e1a0c285269 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:47.888823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28856048-2dde-4bc1-9290-8e1a0c285269 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:47.889307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28856048-2dde-4bc1-9290-8e1a0c285269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:47.905259 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:47.925777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 866/1726] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:56:47 2022] PUT /identity/v3/projects/27b98bc5cfba49fb8a63b1942b3ec09d/users/d4d9687d9cea45f49fab15eb66ac15ab/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:56:47.929401 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-28856048-2dde-4bc1-9290-8e1a0c285269 None None] MFA Rules not processed for user `39d98294ba884280b4971af36a9fbd6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:47.930403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cca01d-93b6-4bfd-b980-ebf6dc3435d4 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:47.930686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cca01d-93b6-4bfd-b980-ebf6dc3435d4 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:47.930980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cca01d-93b6-4bfd-b980-ebf6dc3435d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:47.936846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-28856048-2dde-4bc1-9290-8e1a0c285269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:48.002469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e9cca01d-93b6-4bfd-b980-ebf6dc3435d4 None None] MFA Rules not processed for user `d4d9687d9cea45f49fab15eb66ac15ab`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:48.004408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e9cca01d-93b6-4bfd-b980-ebf6dc3435d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:48.059690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 861/1727] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:48.162003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 867/1728] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:47 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:56:48.177032 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28856048-2dde-4bc1-9290-8e1a0c285269 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:48.180209 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-28856048-2dde-4bc1-9290-8e1a0c285269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:48.272376 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:48.342955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28856048-2dde-4bc1-9290-8e1a0c285269 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:48.358948 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cbabdc-7e3e-4daa-b943-8d7e4041e6a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:48.360836 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbabdc-7e3e-4daa-b943-8d7e4041e6a8 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:48.367863 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbabdc-7e3e-4daa-b943-8d7e4041e6a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:48.367863 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbabdc-7e3e-4daa-b943-8d7e4041e6a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:48.376285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbabdc-7e3e-4daa-b943-8d7e4041e6a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:48.377758 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cbabdc-7e3e-4daa-b943-8d7e4041e6a8 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:48.401127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 862/1729] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:48 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:48.703129 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14152586-084f-499c-be87-e70a8d704493 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:48.703566 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14152586-084f-499c-be87-e70a8d704493 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:48.703844 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14152586-084f-499c-be87-e70a8d704493 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:48.705479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 868/1730] 10.222.0.27 () {54 vars in 912 bytes} [Tue Aug 30 14:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:56:48.715376 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb91b6-b9a8-4ead-a1af-ce21d7c09e89 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:48.715681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb91b6-b9a8-4ead-a1af-ce21d7c09e89 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:48.715943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb91b6-b9a8-4ead-a1af-ce21d7c09e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:48.777862 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-51eb91b6-b9a8-4ead-a1af-ce21d7c09e89 None None] MFA Rules not processed for user `7bff0664e0b44b7c85344a9f3e88f126`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:48.778904 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-51eb91b6-b9a8-4ead-a1af-ce21d7c09e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:48.929069 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 863/1731] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 30 14:56:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:48.952251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14152586-084f-499c-be87-e70a8d704493 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:48.955406 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-14152586-084f-499c-be87-e70a8d704493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:48.978621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb91b6-b9a8-4ead-a1af-ce21d7c09e89 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:48.980502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-51eb91b6-b9a8-4ead-a1af-ce21d7c09e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:48.995501 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14152586-084f-499c-be87-e70a8d704493 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:48.996485 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb91b6-b9a8-4ead-a1af-ce21d7c09e89 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:49.002125 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a05f9d-35b3-491d-ab61-a78891cc851e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:49.003573 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a05f9d-35b3-491d-ab61-a78891cc851e service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:49.004037 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a05f9d-35b3-491d-ab61-a78891cc851e service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:49.004546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a05f9d-35b3-491d-ab61-a78891cc851e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:49.015161 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0dcc73-eeb2-4cea-bf6b-e39cea741d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:49.016240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0dcc73-eeb2-4cea-bf6b-e39cea741d4c service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:49.016841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0dcc73-eeb2-4cea-bf6b-e39cea741d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:49.017405 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0dcc73-eeb2-4cea-bf6b-e39cea741d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:49.035274 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0dcc73-eeb2-4cea-bf6b-e39cea741d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:49.037955 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0dcc73-eeb2-4cea-bf6b-e39cea741d4c service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:49.066322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 869/1732] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:56:49.142835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a05f9d-35b3-491d-ab61-a78891cc851e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:49.145396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a05f9d-35b3-491d-ab61-a78891cc851e service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:49.183820 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 864/1733] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:56:48 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:56:49.668303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:49.878173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cf9a2-93a2-4516-a3c1-3fb80b4a312c None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:49.878459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cf9a2-93a2-4516-a3c1-3fb80b4a312c None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:49.878735 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cf9a2-93a2-4516-a3c1-3fb80b4a312c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:49.928413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-ab8cf9a2-93a2-4516-a3c1-3fb80b4a312c None None] MFA Rules not processed for user `3bd31b8807fc450f8f134417cd535542`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:49.944550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ab8cf9a2-93a2-4516-a3c1-3fb80b4a312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:50.061711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 870/1734] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:50.205493 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:56:50.498066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2092b-bb64-4724-b82d-73661fc24dcd None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:50.498524 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2092b-bb64-4724-b82d-73661fc24dcd None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:50.498977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2092b-bb64-4724-b82d-73661fc24dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:50.544001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0fa2092b-bb64-4724-b82d-73661fc24dcd None None] MFA Rules not processed for user `7dc5cd44b1b34a7ebbdca2c062715431`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:50.545553 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0fa2092b-bb64-4724-b82d-73661fc24dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:50.653031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 865/1735] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:56:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:52.409334 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5704c-0699-45c9-bec7-953526e1bd39 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:52.409750 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5704c-0699-45c9-bec7-953526e1bd39 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:52.410098 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5704c-0699-45c9-bec7-953526e1bd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:52.533609 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-bda5704c-0699-45c9-bec7-953526e1bd39 None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:56:52.535426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-bda5704c-0699-45c9-bec7-953526e1bd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:52.626000 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 871/1736] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 30 14:56:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:56:52.639392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2092b-bb64-4724-b82d-73661fc24dcd None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:52.753943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2092b-bb64-4724-b82d-73661fc24dcd None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:56:52.770061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0e6dd-c3d3-4479-adf2-fc66d63d2d7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:56:52.772203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0e6dd-c3d3-4479-adf2-fc66d63d2d7e service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:56:52.772744 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0e6dd-c3d3-4479-adf2-fc66d63d2d7e service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:56:52.773172 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0e6dd-c3d3-4479-adf2-fc66d63d2d7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:56:52.784246 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a0e6dd-c3d3-4479-adf2-fc66d63d2d7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:56:52.785864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a0e6dd-c3d3-4479-adf2-fc66d63d2d7e service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:56:52.798935 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 866/1737] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 14:56:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:56:59.908860 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5704c-0699-45c9-bec7-953526e1bd39 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:56:59.910429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-bda5704c-0699-45c9-bec7-953526e1bd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:56:59.940255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:00.009329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5704c-0699-45c9-bec7-953526e1bd39 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:00.015238 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b83e4-05a8-4818-a3f0-afd8ac105c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:00.016306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b83e4-05a8-4818-a3f0-afd8ac105c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:00.016851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b83e4-05a8-4818-a3f0-afd8ac105c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:00.017369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b83e4-05a8-4818-a3f0-afd8ac105c1f admin admin] PATH_INFO: `/v3/users/128278a80e304e94b9741a337d8040c0` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:00.018992 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b83e4-05a8-4818-a3f0-afd8ac105c1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=128278a80e304e94b9741a337d8040c0)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:00.022184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b83e4-05a8-4818-a3f0-afd8ac105c1f admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:00.249473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0e6dd-c3d3-4479-adf2-fc66d63d2d7e service swift] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:00.251071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-05a0e6dd-c3d3-4479-adf2-fc66d63d2d7e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:00.267157 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0e6dd-c3d3-4479-adf2-fc66d63d2d7e service swift] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:00.272715 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3eeeb0-7b72-448c-91c1-8373a8919f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:00.273742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3eeeb0-7b72-448c-91c1-8373a8919f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:00.274167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3eeeb0-7b72-448c-91c1-8373a8919f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:00.274569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3eeeb0-7b72-448c-91c1-8373a8919f4d admin admin] PATH_INFO: `/v3/users/a737f87f60fd4b59b252c068467270ca` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:00.275936 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3eeeb0-7b72-448c-91c1-8373a8919f4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a737f87f60fd4b59b252c068467270ca)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:00.279291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3eeeb0-7b72-448c-91c1-8373a8919f4d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:00.298337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:00.355977 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-904b83e4-05a8-4818-a3f0-afd8ac105c1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '128278a80e304e94b9741a337d8040c0'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:00.363549 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-904b83e4-05a8-4818-a3f0-afd8ac105c1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '128278a80e304e94b9741a337d8040c0'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:00.376585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-904b83e4-05a8-4818-a3f0-afd8ac105c1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '128278a80e304e94b9741a337d8040c0'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:00.377324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-904b83e4-05a8-4818-a3f0-afd8ac105c1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '128278a80e304e94b9741a337d8040c0'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:00.378029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-904b83e4-05a8-4818-a3f0-afd8ac105c1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '128278a80e304e94b9741a337d8040c0'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:00.489791 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 872/1738] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:56:59 2022] DELETE /identity/v3/users/128278a80e304e94b9741a337d8040c0 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:00.494758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a441719-834f-49f5-a42a-a1fa1756bd22 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:00.495034 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a441719-834f-49f5-a42a-a1fa1756bd22 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:00.495386 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a441719-834f-49f5-a42a-a1fa1756bd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:00.515651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-3a441719-834f-49f5-a42a-a1fa1756bd22 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:00.516927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-3a441719-834f-49f5-a42a-a1fa1756bd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:00.612493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 873/1739] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:57:00.627693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a441719-834f-49f5-a42a-a1fa1756bd22 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:00.629237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-3a441719-834f-49f5-a42a-a1fa1756bd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:00.644952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0c3eeeb0-7b72-448c-91c1-8373a8919f4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a737f87f60fd4b59b252c068467270ca'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:00.676741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0c3eeeb0-7b72-448c-91c1-8373a8919f4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a737f87f60fd4b59b252c068467270ca'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:00.677501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0c3eeeb0-7b72-448c-91c1-8373a8919f4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a737f87f60fd4b59b252c068467270ca'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:00.697184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:00.713424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a441719-834f-49f5-a42a-a1fa1756bd22 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:00.720237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b313e5-e176-476e-939b-115a93a5e0fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:00.720973 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b313e5-e176-476e-939b-115a93a5e0fd service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:00.721450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b313e5-e176-476e-939b-115a93a5e0fd service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:00.721783 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b313e5-e176-476e-939b-115a93a5e0fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:00.729062 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-72b313e5-e176-476e-939b-115a93a5e0fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:00.745049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b313e5-e176-476e-939b-115a93a5e0fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:00.746570 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b313e5-e176-476e-939b-115a93a5e0fd service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:00.749852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0c3eeeb0-7b72-448c-91c1-8373a8919f4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a737f87f60fd4b59b252c068467270ca'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:00.766402 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0c3eeeb0-7b72-448c-91c1-8373a8919f4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a737f87f60fd4b59b252c068467270ca'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:00.776671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 867/1740] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:00 2022] DELETE /identity/v3/users/a737f87f60fd4b59b252c068467270ca => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:00.777369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 874/1741] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:00.782552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2c49e-42b6-4344-a0c7-9aac9cfadf1b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:00.782841 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2c49e-42b6-4344-a0c7-9aac9cfadf1b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:00.783134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2c49e-42b6-4344-a0c7-9aac9cfadf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:00.822155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ecc2c49e-42b6-4344-a0c7-9aac9cfadf1b None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:00.824825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ecc2c49e-42b6-4344-a0c7-9aac9cfadf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:00.923310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 868/1742] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:00.936485 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b313e5-e176-476e-939b-115a93a5e0fd service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:00.946514 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-72b313e5-e176-476e-939b-115a93a5e0fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:00.986691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b313e5-e176-476e-939b-115a93a5e0fd service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:01.022041 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2c49e-42b6-4344-a0c7-9aac9cfadf1b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:01.023976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ecc2c49e-42b6-4344-a0c7-9aac9cfadf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:01.039170 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2c49e-42b6-4344-a0c7-9aac9cfadf1b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:01.045587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5042b3-c424-4c2f-94ac-342262c64eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:01.046494 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5042b3-c424-4c2f-94ac-342262c64eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:01.046921 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5042b3-c424-4c2f-94ac-342262c64eea admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:01.047369 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5042b3-c424-4c2f-94ac-342262c64eea admin admin] PATH_INFO: `/v3/users/a9330ab132a04fe98091567044c9417e` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:01.048691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5042b3-c424-4c2f-94ac-342262c64eea admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9330ab132a04fe98091567044c9417e)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:01.050913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5042b3-c424-4c2f-94ac-342262c64eea admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:01.118302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854e6fd-8cc7-4bf9-bdd3-067fa813616b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:01.118931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854e6fd-8cc7-4bf9-bdd3-067fa813616b service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:01.119425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854e6fd-8cc7-4bf9-bdd3-067fa813616b service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:01.120157 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854e6fd-8cc7-4bf9-bdd3-067fa813616b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:01.154134 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854e6fd-8cc7-4bf9-bdd3-067fa813616b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:01.156586 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854e6fd-8cc7-4bf9-bdd3-067fa813616b service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:01.180279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 875/1743] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:01.299711 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6c5042b3-c424-4c2f-94ac-342262c64eea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9330ab132a04fe98091567044c9417e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:01.328329 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6c5042b3-c424-4c2f-94ac-342262c64eea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9330ab132a04fe98091567044c9417e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:01.331994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6c5042b3-c424-4c2f-94ac-342262c64eea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9330ab132a04fe98091567044c9417e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:01.440889 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6c5042b3-c424-4c2f-94ac-342262c64eea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9330ab132a04fe98091567044c9417e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:01.456950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6c5042b3-c424-4c2f-94ac-342262c64eea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9330ab132a04fe98091567044c9417e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:01.472924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 869/1744] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:01 2022] DELETE /identity/v3/users/a9330ab132a04fe98091567044c9417e => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:01.477596 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483fed74-27e8-498b-813d-6358036f5857 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:01.477879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483fed74-27e8-498b-813d-6358036f5857 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:01.478404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483fed74-27e8-498b-813d-6358036f5857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:01.501893 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-483fed74-27e8-498b-813d-6358036f5857 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:01.503145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-483fed74-27e8-498b-813d-6358036f5857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:01.623016 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 876/1745] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:01.634741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5042b3-c424-4c2f-94ac-342262c64eea admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:01.636309 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6c5042b3-c424-4c2f-94ac-342262c64eea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:01.642393 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:01.671798 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483fed74-27e8-498b-813d-6358036f5857 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:01.674136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-483fed74-27e8-498b-813d-6358036f5857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:01.706211 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483fed74-27e8-498b-813d-6358036f5857 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:01.713954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c7709-826b-4721-8b70-1d8c86468884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:01.715221 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c7709-826b-4721-8b70-1d8c86468884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:01.715641 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c7709-826b-4721-8b70-1d8c86468884 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:01.716023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c7709-826b-4721-8b70-1d8c86468884 admin admin] PATH_INFO: `/v3/projects/253dee88af3348949a17a4ff0dcc69de` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:01.717563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c7709-826b-4721-8b70-1d8c86468884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=253dee88af3348949a17a4ff0dcc69de)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:01.720398 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c7709-826b-4721-8b70-1d8c86468884 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:01.773041 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5042b3-c424-4c2f-94ac-342262c64eea admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:01.776903 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c4641-a3c3-4a84-a43a-10aa3136c06a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:01.777676 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c4641-a3c3-4a84-a43a-10aa3136c06a service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:01.778097 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c4641-a3c3-4a84-a43a-10aa3136c06a service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:01.778462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c4641-a3c3-4a84-a43a-10aa3136c06a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:01.789437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c4641-a3c3-4a84-a43a-10aa3136c06a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:01.791138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c4641-a3c3-4a84-a43a-10aa3136c06a service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:01.804073 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 870/1746] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:57:01.836291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c4641-a3c3-4a84-a43a-10aa3136c06a service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:01.838043 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-df8c4641-a3c3-4a84-a43a-10aa3136c06a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:01.856426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c4641-a3c3-4a84-a43a-10aa3136c06a service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:01.868920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:01.869563 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:01.871423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:01.871921 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b admin admin] PATH_INFO: `/v3/projects/e00a0929118f464195dd33179049dc09` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:01.873434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e00a0929118f464195dd33179049dc09)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:01.876057 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:01.879664 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e61c7709-826b-4721-8b70-1d8c86468884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '253dee88af3348949a17a4ff0dcc69de', 'request_id': 'req-e61c7709-826b-4721-8b70-1d8c86468884'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:01.887967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e61c7709-826b-4721-8b70-1d8c86468884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '253dee88af3348949a17a4ff0dcc69de', 'request_id': 'req-e61c7709-826b-4721-8b70-1d8c86468884'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:01.967234 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e61c7709-826b-4721-8b70-1d8c86468884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '253dee88af3348949a17a4ff0dcc69de', 'request_id': 'req-e61c7709-826b-4721-8b70-1d8c86468884'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.007336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e61c7709-826b-4721-8b70-1d8c86468884 admin admin] The token cache is being invalidate because project 253dee88af3348949a17a4ff0dcc69de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:02.008108 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e61c7709-826b-4721-8b70-1d8c86468884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.011490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e61c7709-826b-4721-8b70-1d8c86468884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.015430 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 877/1747] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:01 2022] DELETE /identity/v3/projects/253dee88af3348949a17a4ff0dcc69de => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:02.020798 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c7709-826b-4721-8b70-1d8c86468884 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:02.024351 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e61c7709-826b-4721-8b70-1d8c86468884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:02.048406 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:02.075685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e00a0929118f464195dd33179049dc09', 'request_id': 'req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.083151 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e00a0929118f464195dd33179049dc09', 'request_id': 'req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.112526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e00a0929118f464195dd33179049dc09', 'request_id': 'req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.124080 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c7709-826b-4721-8b70-1d8c86468884 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:02.192021 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b admin admin] The token cache is being invalidate because project e00a0929118f464195dd33179049dc09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:02.192447 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.193071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.193814 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5196b66-c086-4fe1-9960-d9292ccf1220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:02.194704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5196b66-c086-4fe1-9960-d9292ccf1220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:02.195006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5196b66-c086-4fe1-9960-d9292ccf1220 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:02.195292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5196b66-c086-4fe1-9960-d9292ccf1220 admin admin] PATH_INFO: `/v3/users/ce515bc9fbeb4be78038c92fbaa71ba5` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:02.196544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5196b66-c086-4fe1-9960-d9292ccf1220 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce515bc9fbeb4be78038c92fbaa71ba5)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:02.198626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5196b66-c086-4fe1-9960-d9292ccf1220 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:02.198750 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 871/1748] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:01 2022] DELETE /identity/v3/projects/e00a0929118f464195dd33179049dc09 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:02.202620 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:02.207870 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:02.226821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111e622-f2ea-4287-9a6a-bcc2e4ca731b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:02.232244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4348ec-25b2-454b-b5e8-313e38e74bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:02.232898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4348ec-25b2-454b-b5e8-313e38e74bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:02.233166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4348ec-25b2-454b-b5e8-313e38e74bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:02.233454 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4348ec-25b2-454b-b5e8-313e38e74bd9 admin admin] PATH_INFO: `/v3/users/e9bb53cc6bf74405a3d29d5fc11a21ac` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:02.234858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4348ec-25b2-454b-b5e8-313e38e74bd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9bb53cc6bf74405a3d29d5fc11a21ac)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:02.236843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4348ec-25b2-454b-b5e8-313e38e74bd9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:02.430795 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c5196b66-c086-4fe1-9960-d9292ccf1220 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce515bc9fbeb4be78038c92fbaa71ba5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.438390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c5196b66-c086-4fe1-9960-d9292ccf1220 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce515bc9fbeb4be78038c92fbaa71ba5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.456748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c5196b66-c086-4fe1-9960-d9292ccf1220 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce515bc9fbeb4be78038c92fbaa71ba5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.456748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c5196b66-c086-4fe1-9960-d9292ccf1220 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce515bc9fbeb4be78038c92fbaa71ba5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.456748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c5196b66-c086-4fe1-9960-d9292ccf1220 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce515bc9fbeb4be78038c92fbaa71ba5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.500586 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1b4348ec-25b2-454b-b5e8-313e38e74bd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9bb53cc6bf74405a3d29d5fc11a21ac'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.508276 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1b4348ec-25b2-454b-b5e8-313e38e74bd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9bb53cc6bf74405a3d29d5fc11a21ac'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.512823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1b4348ec-25b2-454b-b5e8-313e38e74bd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9bb53cc6bf74405a3d29d5fc11a21ac'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.569979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 878/1749] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:02 2022] DELETE /identity/v3/users/ce515bc9fbeb4be78038c92fbaa71ba5 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:02.570812 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5196b66-c086-4fe1-9960-d9292ccf1220 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:02.572182 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c5196b66-c086-4fe1-9960-d9292ccf1220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:02.608084 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:02.646782 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5196b66-c086-4fe1-9960-d9292ccf1220 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:02.652703 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a21ac0-f8ea-405d-8121-31f996baf913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:02.653360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a21ac0-f8ea-405d-8121-31f996baf913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:02.653641 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a21ac0-f8ea-405d-8121-31f996baf913 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:02.653932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a21ac0-f8ea-405d-8121-31f996baf913 admin admin] PATH_INFO: `/v3/projects/773c9f06b9264acab70e57aabcff1b7e` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:02.655268 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a21ac0-f8ea-405d-8121-31f996baf913 admin admin] RBAC: Authorizing `identity:delete_project(project_id=773c9f06b9264acab70e57aabcff1b7e)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:02.657152 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a21ac0-f8ea-405d-8121-31f996baf913 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:02.711890 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1b4348ec-25b2-454b-b5e8-313e38e74bd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9bb53cc6bf74405a3d29d5fc11a21ac'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.725491 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1b4348ec-25b2-454b-b5e8-313e38e74bd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9bb53cc6bf74405a3d29d5fc11a21ac'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.727669 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:02.731960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 872/1750] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:02 2022] DELETE /identity/v3/users/e9bb53cc6bf74405a3d29d5fc11a21ac => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:02.773820 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d3321-3edf-4608-9367-e6540472295a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:02.774160 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d3321-3edf-4608-9367-e6540472295a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:02.774438 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d3321-3edf-4608-9367-e6540472295a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:02.808054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-489d3321-3edf-4608-9367-e6540472295a None None] MFA Rules not processed for user `7bff0664e0b44b7c85344a9f3e88f126`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:02.809643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-489d3321-3edf-4608-9367-e6540472295a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:02.827061 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b9a21ac0-f8ea-405d-8121-31f996baf913 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '773c9f06b9264acab70e57aabcff1b7e', 'request_id': 'req-b9a21ac0-f8ea-405d-8121-31f996baf913'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.833038 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b9a21ac0-f8ea-405d-8121-31f996baf913 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '773c9f06b9264acab70e57aabcff1b7e', 'request_id': 'req-b9a21ac0-f8ea-405d-8121-31f996baf913'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.887422 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 873/1751] 10.222.0.59 () {58 vars in 1015 bytes} [Tue Aug 30 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:57:02.906130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d3321-3edf-4608-9367-e6540472295a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:02.908594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-489d3321-3edf-4608-9367-e6540472295a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:02.922726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d3321-3edf-4608-9367-e6540472295a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:02.927157 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea02951-37b9-4605-84f7-dd7a9cb82c11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:02.928344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea02951-37b9-4605-84f7-dd7a9cb82c11 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:02.928629 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea02951-37b9-4605-84f7-dd7a9cb82c11 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:02.928891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea02951-37b9-4605-84f7-dd7a9cb82c11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:02.936891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea02951-37b9-4605-84f7-dd7a9cb82c11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:02.939258 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea02951-37b9-4605-84f7-dd7a9cb82c11 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:02.943567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b9a21ac0-f8ea-405d-8121-31f996baf913 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '773c9f06b9264acab70e57aabcff1b7e', 'request_id': 'req-b9a21ac0-f8ea-405d-8121-31f996baf913'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.950761 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 874/1752] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:02.962785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b9a21ac0-f8ea-405d-8121-31f996baf913 admin admin] The token cache is being invalidate because project 773c9f06b9264acab70e57aabcff1b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:02.962785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b9a21ac0-f8ea-405d-8121-31f996baf913 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.963669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b9a21ac0-f8ea-405d-8121-31f996baf913 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:02.967148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 879/1753] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:02 2022] DELETE /identity/v3/projects/773c9f06b9264acab70e57aabcff1b7e => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:02.977295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea02951-37b9-4605-84f7-dd7a9cb82c11 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:02.980333 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bea02951-37b9-4605-84f7-dd7a9cb82c11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:03.147328 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea02951-37b9-4605-84f7-dd7a9cb82c11 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:03.152544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027e554-f599-4240-b5ca-ebffc28b21d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:03.153572 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027e554-f599-4240-b5ca-ebffc28b21d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:03.153956 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027e554-f599-4240-b5ca-ebffc28b21d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:03.154360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027e554-f599-4240-b5ca-ebffc28b21d3 admin admin] PATH_INFO: `/v3/users/d4d9687d9cea45f49fab15eb66ac15ab` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:03.157398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027e554-f599-4240-b5ca-ebffc28b21d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4d9687d9cea45f49fab15eb66ac15ab)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:03.161075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027e554-f599-4240-b5ca-ebffc28b21d3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:03.426056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8027e554-f599-4240-b5ca-ebffc28b21d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4d9687d9cea45f49fab15eb66ac15ab'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:03.463739 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8027e554-f599-4240-b5ca-ebffc28b21d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4d9687d9cea45f49fab15eb66ac15ab'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:03.464699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8027e554-f599-4240-b5ca-ebffc28b21d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4d9687d9cea45f49fab15eb66ac15ab'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:03.541781 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a21ac0-f8ea-405d-8121-31f996baf913 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:03.544170 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b9a21ac0-f8ea-405d-8121-31f996baf913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:03.552579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8027e554-f599-4240-b5ca-ebffc28b21d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4d9687d9cea45f49fab15eb66ac15ab'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:03.578175 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8027e554-f599-4240-b5ca-ebffc28b21d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4d9687d9cea45f49fab15eb66ac15ab'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:03.586561 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a21ac0-f8ea-405d-8121-31f996baf913 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:03.594285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 875/1754] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:02 2022] DELETE /identity/v3/users/d4d9687d9cea45f49fab15eb66ac15ab => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:03.600617 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670386d3-e323-47ac-9f14-aafd38bb687d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:03.603841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670386d3-e323-47ac-9f14-aafd38bb687d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:03.604131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670386d3-e323-47ac-9f14-aafd38bb687d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:03.604407 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670386d3-e323-47ac-9f14-aafd38bb687d admin admin] PATH_INFO: `/v3/projects/155748f5300142d396c4724884a4897b` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:03.605966 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670386d3-e323-47ac-9f14-aafd38bb687d admin admin] RBAC: Authorizing `identity:delete_project(project_id=155748f5300142d396c4724884a4897b)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:03.615142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670386d3-e323-47ac-9f14-aafd38bb687d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:03.778229 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-670386d3-e323-47ac-9f14-aafd38bb687d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '155748f5300142d396c4724884a4897b', 'request_id': 'req-670386d3-e323-47ac-9f14-aafd38bb687d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:03.786492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-670386d3-e323-47ac-9f14-aafd38bb687d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '155748f5300142d396c4724884a4897b', 'request_id': 'req-670386d3-e323-47ac-9f14-aafd38bb687d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:03.845525 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027e554-f599-4240-b5ca-ebffc28b21d3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:03.846999 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8027e554-f599-4240-b5ca-ebffc28b21d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:03.853204 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-670386d3-e323-47ac-9f14-aafd38bb687d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '155748f5300142d396c4724884a4897b', 'request_id': 'req-670386d3-e323-47ac-9f14-aafd38bb687d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:03.886708 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-670386d3-e323-47ac-9f14-aafd38bb687d admin admin] The token cache is being invalidate because project 155748f5300142d396c4724884a4897b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:03.887217 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-670386d3-e323-47ac-9f14-aafd38bb687d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:03.889761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-670386d3-e323-47ac-9f14-aafd38bb687d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:03.894109 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 880/1755] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:03 2022] DELETE /identity/v3/projects/155748f5300142d396c4724884a4897b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:03.905660 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856bbcf-3c25-492a-9cf2-2ab974a8e131 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:03.906260 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856bbcf-3c25-492a-9cf2-2ab974a8e131 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:03.906882 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856bbcf-3c25-492a-9cf2-2ab974a8e131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:03.962234 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-c856bbcf-3c25-492a-9cf2-2ab974a8e131 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:03.964157 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c856bbcf-3c25-492a-9cf2-2ab974a8e131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:03.983847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027e554-f599-4240-b5ca-ebffc28b21d3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:04.073356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482649a-eacd-4c9b-aaec-fed24ae2122e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:04.075163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482649a-eacd-4c9b-aaec-fed24ae2122e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:04.075457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482649a-eacd-4c9b-aaec-fed24ae2122e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:04.075735 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482649a-eacd-4c9b-aaec-fed24ae2122e admin admin] PATH_INFO: `/v3/projects/8739a250ea1f400eb3fb88c72c312ac4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:04.078490 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482649a-eacd-4c9b-aaec-fed24ae2122e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8739a250ea1f400eb3fb88c72c312ac4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:04.080772 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482649a-eacd-4c9b-aaec-fed24ae2122e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:04.133263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:04.177239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 881/1756] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:57:04.186528 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d53613-2540-4993-ac1b-2fda4ae9edc2 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:04.186967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d53613-2540-4993-ac1b-2fda4ae9edc2 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:04.187386 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d53613-2540-4993-ac1b-2fda4ae9edc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:04.216723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e3d53613-2540-4993-ac1b-2fda4ae9edc2 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:04.218513 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e3d53613-2540-4993-ac1b-2fda4ae9edc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:04.255907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 882/1757] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:04.260153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d53613-2540-4993-ac1b-2fda4ae9edc2 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:04.276248 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7482649a-eacd-4c9b-aaec-fed24ae2122e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8739a250ea1f400eb3fb88c72c312ac4', 'request_id': 'req-7482649a-eacd-4c9b-aaec-fed24ae2122e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:04.281102 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d53613-2540-4993-ac1b-2fda4ae9edc2 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:04.287645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7482649a-eacd-4c9b-aaec-fed24ae2122e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8739a250ea1f400eb3fb88c72c312ac4', 'request_id': 'req-7482649a-eacd-4c9b-aaec-fed24ae2122e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:04.294793 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:04.303848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7482649a-eacd-4c9b-aaec-fed24ae2122e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8739a250ea1f400eb3fb88c72c312ac4', 'request_id': 'req-7482649a-eacd-4c9b-aaec-fed24ae2122e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:04.358836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c8bec-617d-46f1-bcb0-807bc3c2156a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:04.359539 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c8bec-617d-46f1-bcb0-807bc3c2156a admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:04.359817 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c8bec-617d-46f1-bcb0-807bc3c2156a admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:04.360087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c8bec-617d-46f1-bcb0-807bc3c2156a admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:04.360747 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c8bec-617d-46f1-bcb0-807bc3c2156a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:57:04.361038 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c8bec-617d-46f1-bcb0-807bc3c2156a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:04.362720 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c8bec-617d-46f1-bcb0-807bc3c2156a admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:04.397018 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 883/1758] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:57:04.398102 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c8bec-617d-46f1-bcb0-807bc3c2156a admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:04.411649 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7482649a-eacd-4c9b-aaec-fed24ae2122e admin admin] The token cache is being invalidate because project 8739a250ea1f400eb3fb88c72c312ac4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:04.412751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7482649a-eacd-4c9b-aaec-fed24ae2122e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:04.413569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7482649a-eacd-4c9b-aaec-fed24ae2122e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:04.416147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 876/1759] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:03 2022] DELETE /identity/v3/projects/8739a250ea1f400eb3fb88c72c312ac4 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:04.420435 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c8bec-617d-46f1-bcb0-807bc3c2156a admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:04.421224 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec4d11-f7e3-4a44-b397-198384fd232e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:04.421790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec4d11-f7e3-4a44-b397-198384fd232e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:04.422210 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec4d11-f7e3-4a44-b397-198384fd232e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:04.435929 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e6912-9fd4-41e9-8d10-e1375aeed6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:04.437388 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e6912-9fd4-41e9-8d10-e1375aeed6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:04.437775 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e6912-9fd4-41e9-8d10-e1375aeed6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:04.439446 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e6912-9fd4-41e9-8d10-e1375aeed6f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:04.439446 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e6912-9fd4-41e9-8d10-e1375aeed6f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:04.441476 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e6912-9fd4-41e9-8d10-e1375aeed6f1 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:04.474058 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-79ec4d11-f7e3-4a44-b397-198384fd232e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:04.476698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-79ec4d11-f7e3-4a44-b397-198384fd232e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:04.516485 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 877/1760] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:04.518602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34dbd2-e458-4942-9166-f34372c66a76 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:04.518963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34dbd2-e458-4942-9166-f34372c66a76 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:04.519348 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34dbd2-e458-4942-9166-f34372c66a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:04.557364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4b34dbd2-e458-4942-9166-f34372c66a76 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:04.558710 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4b34dbd2-e458-4942-9166-f34372c66a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:04.568844 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 878/1761] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:04.584663 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 884/1762] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:04 2022] POST /identity/v3/projects => generated 375 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:04.585191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34dbd2-e458-4942-9166-f34372c66a76 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:04.586201 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e6912-9fd4-41e9-8d10-e1375aeed6f1 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:04.591817 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-5c4e6912-9fd4-41e9-8d10-e1375aeed6f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:04.596003 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34dbd2-e458-4942-9166-f34372c66a76 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:04.687844 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66418dbe-9848-447e-beee-06cd56e866ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:04.688480 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66418dbe-9848-447e-beee-06cd56e866ae admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:04.688817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66418dbe-9848-447e-beee-06cd56e866ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:04.689097 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66418dbe-9848-447e-beee-06cd56e866ae admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:04.690209 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66418dbe-9848-447e-beee-06cd56e866ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:57:04.690533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66418dbe-9848-447e-beee-06cd56e866ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:04.692366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66418dbe-9848-447e-beee-06cd56e866ae admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:04.716096 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 879/1763] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:57:04.719834 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66418dbe-9848-447e-beee-06cd56e866ae admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:04.734324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66418dbe-9848-447e-beee-06cd56e866ae admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:04.742886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b977c14-9363-4bd7-9492-47cdebba8af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:04.743868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b977c14-9363-4bd7-9492-47cdebba8af7 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:04.744393 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b977c14-9363-4bd7-9492-47cdebba8af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:04.744954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b977c14-9363-4bd7-9492-47cdebba8af7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:04.747603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b977c14-9363-4bd7-9492-47cdebba8af7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:04.750948 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b977c14-9363-4bd7-9492-47cdebba8af7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:04.781810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e6912-9fd4-41e9-8d10-e1375aeed6f1 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:04.789548 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9230227-0a84-4804-93d1-005887f0dfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:04.790765 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9230227-0a84-4804-93d1-005887f0dfc0 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:04.791809 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9230227-0a84-4804-93d1-005887f0dfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:04.792502 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9230227-0a84-4804-93d1-005887f0dfc0 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:04.793347 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9230227-0a84-4804-93d1-005887f0dfc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:04.796535 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9230227-0a84-4804-93d1-005887f0dfc0 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:04.907528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 880/1764] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:04 2022] POST /identity/v3/projects => generated 365 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:04.912692 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b977c14-9363-4bd7-9492-47cdebba8af7 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:04.990210 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 885/1765] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:04 2022] POST /identity/v3/users => generated 288 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:04.996223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9230227-0a84-4804-93d1-005887f0dfc0 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:05.018852 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9230227-0a84-4804-93d1-005887f0dfc0 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:05.025118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d55bc5-e765-4e98-8d43-85d9f726c54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:05.026138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d55bc5-e765-4e98-8d43-85d9f726c54a admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:05.026138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d55bc5-e765-4e98-8d43-85d9f726c54a admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:05.026669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d55bc5-e765-4e98-8d43-85d9f726c54a admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:05.027396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d55bc5-e765-4e98-8d43-85d9f726c54a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:05.029002 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d55bc5-e765-4e98-8d43-85d9f726c54a admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:05.035698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b977c14-9363-4bd7-9492-47cdebba8af7 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:05.044324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 886/1766] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:05.048291 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d55bc5-e765-4e98-8d43-85d9f726c54a admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:05.051798 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fd75b-91b4-4b93-8550-57d08ced333e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:05.053067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fd75b-91b4-4b93-8550-57d08ced333e admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:05.053356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fd75b-91b4-4b93-8550-57d08ced333e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:05.053617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fd75b-91b4-4b93-8550-57d08ced333e admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:05.055076 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fd75b-91b4-4b93-8550-57d08ced333e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:05.057080 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fd75b-91b4-4b93-8550-57d08ced333e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:05.061916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d55bc5-e765-4e98-8d43-85d9f726c54a admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:05.082470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2909bbf-8af8-4f64-8ea8-e2c0a4afd523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:05.094443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2909bbf-8af8-4f64-8ea8-e2c0a4afd523 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:05.095190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2909bbf-8af8-4f64-8ea8-e2c0a4afd523 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:05.095495 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2909bbf-8af8-4f64-8ea8-e2c0a4afd523 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:05.096637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2909bbf-8af8-4f64-8ea8-e2c0a4afd523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:05.104600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2909bbf-8af8-4f64-8ea8-e2c0a4afd523 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:05.127326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 887/1767] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:57:05.128439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2909bbf-8af8-4f64-8ea8-e2c0a4afd523 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:05.139698 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2909bbf-8af8-4f64-8ea8-e2c0a4afd523 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:05.148539 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222f182-46e3-4443-99e6-915d05314a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:05.149288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222f182-46e3-4443-99e6-915d05314a6c admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:05.149560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222f182-46e3-4443-99e6-915d05314a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:05.150036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222f182-46e3-4443-99e6-915d05314a6c admin admin] PATH_INFO: `/v3/projects/bc032c1c32824a83a80f852244dfeff4/users/06fe98653eac47cdb31cdf0928b5ad5a/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:05.176353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222f182-46e3-4443-99e6-915d05314a6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc032c1c32824a83a80f852244dfeff4, user_id=06fe98653eac47cdb31cdf0928b5ad5a, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:05.197930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 881/1768] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:04 2022] POST /identity/v3/users => generated 283 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:05.197930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fd75b-91b4-4b93-8550-57d08ced333e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:05.199723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222f182-46e3-4443-99e6-915d05314a6c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:05.204442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a39fd75b-91b4-4b93-8550-57d08ced333e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:05.222896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fd75b-91b4-4b93-8550-57d08ced333e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:05.231811 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6ce35-09f4-4974-9239-0d10e9460289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:05.233100 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6ce35-09f4-4974-9239-0d10e9460289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:05.234052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6ce35-09f4-4974-9239-0d10e9460289 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:05.234568 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6ce35-09f4-4974-9239-0d10e9460289 admin admin] PATH_INFO: `/v3/projects/27b98bc5cfba49fb8a63b1942b3ec09d` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:05.236987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6ce35-09f4-4974-9239-0d10e9460289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b98bc5cfba49fb8a63b1942b3ec09d)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:05.239481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6ce35-09f4-4974-9239-0d10e9460289 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:05.314819 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 888/1769] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:05 2022] PUT /identity/v3/projects/bc032c1c32824a83a80f852244dfeff4/users/06fe98653eac47cdb31cdf0928b5ad5a/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:05.315909 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222f182-46e3-4443-99e6-915d05314a6c admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:05.448695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5b6ce35-09f4-4974-9239-0d10e9460289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b98bc5cfba49fb8a63b1942b3ec09d', 'request_id': 'req-d5b6ce35-09f4-4974-9239-0d10e9460289'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:05.468380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222f182-46e3-4443-99e6-915d05314a6c admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:05.470195 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5b6ce35-09f4-4974-9239-0d10e9460289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b98bc5cfba49fb8a63b1942b3ec09d', 'request_id': 'req-d5b6ce35-09f4-4974-9239-0d10e9460289'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:05.490454 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5b6ce35-09f4-4974-9239-0d10e9460289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b98bc5cfba49fb8a63b1942b3ec09d', 'request_id': 'req-d5b6ce35-09f4-4974-9239-0d10e9460289'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:05.592941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbb66d-ef66-4846-83aa-1241430f6c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:05.593553 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5b6ce35-09f4-4974-9239-0d10e9460289 admin admin] The token cache is being invalidate because project 27b98bc5cfba49fb8a63b1942b3ec09d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:05.593961 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbb66d-ef66-4846-83aa-1241430f6c11 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:05.594033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5b6ce35-09f4-4974-9239-0d10e9460289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:05.594100 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbb66d-ef66-4846-83aa-1241430f6c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:05.594100 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbb66d-ef66-4846-83aa-1241430f6c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:05.594725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbb66d-ef66-4846-83aa-1241430f6c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:05.595279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5b6ce35-09f4-4974-9239-0d10e9460289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:05.596679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbb66d-ef66-4846-83aa-1241430f6c11 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:05.603858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 882/1770] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:05 2022] DELETE /identity/v3/projects/27b98bc5cfba49fb8a63b1942b3ec09d => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:05.603858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bee22-dafd-4128-8228-1e06ecbcbd80 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:05.603858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bee22-dafd-4128-8228-1e06ecbcbd80 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:05.603858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bee22-dafd-4128-8228-1e06ecbcbd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:05.623769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 889/1771] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:05.628333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b260655-3ec2-4b09-8baf-4318c771c9c6 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:05.628622 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b260655-3ec2-4b09-8baf-4318c771c9c6 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:05.628899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b260655-3ec2-4b09-8baf-4318c771c9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:05.652455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-9a5bee22-dafd-4128-8228-1e06ecbcbd80 None None] MFA Rules not processed for user `06fe98653eac47cdb31cdf0928b5ad5a`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:05.656452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-9b260655-3ec2-4b09-8baf-4318c771c9c6 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:05.656994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9a5bee22-dafd-4128-8228-1e06ecbcbd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:05.657649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9b260655-3ec2-4b09-8baf-4318c771c9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:05.668995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 890/1772] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:05.670777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a1d6c-2d5f-4d63-85a6-69e80783fa81 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:05.671356 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a1d6c-2d5f-4d63-85a6-69e80783fa81 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:05.671796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a1d6c-2d5f-4d63-85a6-69e80783fa81 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:05.673508 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 891/1773] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 30 14:57:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:57:05.674342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a1d6c-2d5f-4d63-85a6-69e80783fa81 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:05.676740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-fb0a1d6c-2d5f-4d63-85a6-69e80783fa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:05.695569 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a1d6c-2d5f-4d63-85a6-69e80783fa81 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:05.700772 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8707a8d-7fa4-4df6-b8e4-29b02c877be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:05.701570 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8707a8d-7fa4-4df6-b8e4-29b02c877be5 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:05.702117 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8707a8d-7fa4-4df6-b8e4-29b02c877be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:05.702529 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8707a8d-7fa4-4df6-b8e4-29b02c877be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:05.703364 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8707a8d-7fa4-4df6-b8e4-29b02c877be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:05.705216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8707a8d-7fa4-4df6-b8e4-29b02c877be5 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:05.729246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 892/1774] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:05.729246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69416b75-a231-42fd-9f33-2b0582bc4b3c None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:05.729246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69416b75-a231-42fd-9f33-2b0582bc4b3c None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:05.729246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69416b75-a231-42fd-9f33-2b0582bc4b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:05.778322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-69416b75-a231-42fd-9f33-2b0582bc4b3c None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:05.779572 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-69416b75-a231-42fd-9f33-2b0582bc4b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:05.790273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 883/1775] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:05.790273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624b524-7f6a-4624-ac03-08a152b9e4be None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:05.790273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624b524-7f6a-4624-ac03-08a152b9e4be None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:05.790273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624b524-7f6a-4624-ac03-08a152b9e4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:05.801274 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 893/1776] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:05.801274 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69416b75-a231-42fd-9f33-2b0582bc4b3c None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:05.811627 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69416b75-a231-42fd-9f33-2b0582bc4b3c None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:05.817952 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdd711-f9ff-4117-8c9c-c69664e0a225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:05.817952 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdd711-f9ff-4117-8c9c-c69664e0a225 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:05.818697 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdd711-f9ff-4117-8c9c-c69664e0a225 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:05.818697 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdd711-f9ff-4117-8c9c-c69664e0a225 admin admin] PATH_INFO: `/v3/projects/76b746e35f9c4268bb4c2bdfc9c02ee6/users/0baa7e10632848b39a55f7e69abfe603/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:05.839507 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-8624b524-7f6a-4624-ac03-08a152b9e4be None None] MFA Rules not processed for user `5b6e4c5bf2a54b28a55e5ae74b90ab4f`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:05.840851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8624b524-7f6a-4624-ac03-08a152b9e4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:05.850656 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdd711-f9ff-4117-8c9c-c69664e0a225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76b746e35f9c4268bb4c2bdfc9c02ee6, user_id=0baa7e10632848b39a55f7e69abfe603, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:05.863076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdd711-f9ff-4117-8c9c-c69664e0a225 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:05.967500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 894/1777] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:05 2022] PUT /identity/v3/projects/76b746e35f9c4268bb4c2bdfc9c02ee6/users/0baa7e10632848b39a55f7e69abfe603/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:05.968671 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdd711-f9ff-4117-8c9c-c69664e0a225 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:05.981627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 884/1778] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 30 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:05.981627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624b524-7f6a-4624-ac03-08a152b9e4be None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:05.983086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-50cdd711-f9ff-4117-8c9c-c69664e0a225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:05.997166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624b524-7f6a-4624-ac03-08a152b9e4be None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:06.072888 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdd711-f9ff-4117-8c9c-c69664e0a225 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:06.074844 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48eea0b-52b0-4ea8-aab4-9f571a347336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:06.075692 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eea0b-52b0-4ea8-aab4-9f571a347336 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:06.076096 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eea0b-52b0-4ea8-aab4-9f571a347336 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:06.076478 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eea0b-52b0-4ea8-aab4-9f571a347336 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:06.077274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48eea0b-52b0-4ea8-aab4-9f571a347336 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:57:06.077679 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48eea0b-52b0-4ea8-aab4-9f571a347336 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:06.078160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3f013-f4bc-4fdd-80f3-9f74da3a5185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:06.078766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3f013-f4bc-4fdd-80f3-9f74da3a5185 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:06.079042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3f013-f4bc-4fdd-80f3-9f74da3a5185 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:06.079365 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3f013-f4bc-4fdd-80f3-9f74da3a5185 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:06.079860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48eea0b-52b0-4ea8-aab4-9f571a347336 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:06.079990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3f013-f4bc-4fdd-80f3-9f74da3a5185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:06.082303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3f013-f4bc-4fdd-80f3-9f74da3a5185 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:06.093271 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 885/1779] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:57:06.094455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb6420-a4d9-4bfd-8071-d87453b7fdc9 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:06.094815 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb6420-a4d9-4bfd-8071-d87453b7fdc9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:06.095215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb6420-a4d9-4bfd-8071-d87453b7fdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:06.126990 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0dcb6420-a4d9-4bfd-8071-d87453b7fdc9 None None] MFA Rules not processed for user `0baa7e10632848b39a55f7e69abfe603`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:06.128244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0dcb6420-a4d9-4bfd-8071-d87453b7fdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:06.172724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 895/1780] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:05 2022] POST /identity/v3/projects => generated 375 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:06.173685 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3f013-f4bc-4fdd-80f3-9f74da3a5185 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:06.178777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8bb3f013-f4bc-4fdd-80f3-9f74da3a5185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:06.219937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 886/1781] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:06.221264 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb6420-a4d9-4bfd-8071-d87453b7fdc9 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:06.245024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb6420-a4d9-4bfd-8071-d87453b7fdc9 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:06.292364 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3f013-f4bc-4fdd-80f3-9f74da3a5185 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:06.299197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17345823-1948-4831-9547-c816f9107e8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:06.300061 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17345823-1948-4831-9547-c816f9107e8f service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:06.300456 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17345823-1948-4831-9547-c816f9107e8f service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:06.300830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17345823-1948-4831-9547-c816f9107e8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:06.311003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17345823-1948-4831-9547-c816f9107e8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:06.312758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17345823-1948-4831-9547-c816f9107e8f service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:06.324864 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 896/1782] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:06.325758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17345823-1948-4831-9547-c816f9107e8f service nova] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:06.340435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb543f66-220a-44d6-b613-2085b7bb491a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:06.341771 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb543f66-220a-44d6-b613-2085b7bb491a admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:06.343136 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb543f66-220a-44d6-b613-2085b7bb491a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:06.343467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb543f66-220a-44d6-b613-2085b7bb491a admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:06.344126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb543f66-220a-44d6-b613-2085b7bb491a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:06.355609 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb543f66-220a-44d6-b613-2085b7bb491a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:06.418598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17345823-1948-4831-9547-c816f9107e8f service nova] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:06.426232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0433dc9-6e61-4204-9750-4b95a233253c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:06.426919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0433dc9-6e61-4204-9750-4b95a233253c admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:06.427342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0433dc9-6e61-4204-9750-4b95a233253c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:06.427634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0433dc9-6e61-4204-9750-4b95a233253c admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:06.428375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0433dc9-6e61-4204-9750-4b95a233253c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:06.430363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0433dc9-6e61-4204-9750-4b95a233253c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:06.489109 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 887/1783] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:06 2022] POST /identity/v3/projects => generated 377 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:06.490085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb543f66-220a-44d6-b613-2085b7bb491a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:06.536843 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 897/1784] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:06 2022] POST /identity/v3/users => generated 294 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:06.537866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0433dc9-6e61-4204-9750-4b95a233253c admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:06.555225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0433dc9-6e61-4204-9750-4b95a233253c admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:06.597708 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb543f66-220a-44d6-b613-2085b7bb491a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:06.613670 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6734fd-81f7-4b05-9eeb-0504c9b64142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:06.614018 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6734fd-81f7-4b05-9eeb-0504c9b64142 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:06.614481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6734fd-81f7-4b05-9eeb-0504c9b64142 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:06.614585 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6734fd-81f7-4b05-9eeb-0504c9b64142 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:06.619048 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6734fd-81f7-4b05-9eeb-0504c9b64142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:06.621085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6734fd-81f7-4b05-9eeb-0504c9b64142 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:06.640696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a107b-fc90-4180-8375-0f0a7fa5fa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:06.641411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a107b-fc90-4180-8375-0f0a7fa5fa26 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:06.641756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a107b-fc90-4180-8375-0f0a7fa5fa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:06.642723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a107b-fc90-4180-8375-0f0a7fa5fa26 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:06.643562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a107b-fc90-4180-8375-0f0a7fa5fa26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:06.649480 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a107b-fc90-4180-8375-0f0a7fa5fa26 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:06.675039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:06.774097 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 898/1785] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:06 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:06.774097 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a107b-fc90-4180-8375-0f0a7fa5fa26 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:06.786199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a107b-fc90-4180-8375-0f0a7fa5fa26 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:06.792649 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 888/1786] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:06 2022] POST /identity/v3/projects => generated 361 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:06.793303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779caa7-d6e8-44cd-acdf-5cf1b747b081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:06.793598 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6734fd-81f7-4b05-9eeb-0504c9b64142 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:06.795542 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779caa7-d6e8-44cd-acdf-5cf1b747b081 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:06.796291 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779caa7-d6e8-44cd-acdf-5cf1b747b081 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:06.796831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779caa7-d6e8-44cd-acdf-5cf1b747b081 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:06.797669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779caa7-d6e8-44cd-acdf-5cf1b747b081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:06.799631 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779caa7-d6e8-44cd-acdf-5cf1b747b081 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:06.815185 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 899/1787] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:06.816025 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779caa7-d6e8-44cd-acdf-5cf1b747b081 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:06.850413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779caa7-d6e8-44cd-acdf-5cf1b747b081 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:06.910808 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:06.951075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6734fd-81f7-4b05-9eeb-0504c9b64142 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:06.971440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ed123-4a8a-4f86-b862-33d42746d9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:06.975603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ed123-4a8a-4f86-b862-33d42746d9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:06.975969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ed123-4a8a-4f86-b862-33d42746d9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:06.976321 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ed123-4a8a-4f86-b862-33d42746d9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:06.977576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ed123-4a8a-4f86-b862-33d42746d9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:06.980024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ed123-4a8a-4f86-b862-33d42746d9c8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:06.986851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377047c6-c0cf-4046-bfd7-5349d2d8da3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:06.991053 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377047c6-c0cf-4046-bfd7-5349d2d8da3f admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:06.991374 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377047c6-c0cf-4046-bfd7-5349d2d8da3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:06.993105 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377047c6-c0cf-4046-bfd7-5349d2d8da3f admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:06.993790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377047c6-c0cf-4046-bfd7-5349d2d8da3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:07.010625 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377047c6-c0cf-4046-bfd7-5349d2d8da3f admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:07.014370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 889/1788] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:07.015426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ed123-4a8a-4f86-b862-33d42746d9c8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:07.036566 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ed123-4a8a-4f86-b862-33d42746d9c8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:07.054356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50328979-e9fb-4cfa-8888-b20489911b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:07.054356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50328979-e9fb-4cfa-8888-b20489911b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:07.054356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50328979-e9fb-4cfa-8888-b20489911b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:07.054598 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50328979-e9fb-4cfa-8888-b20489911b37 admin admin] PATH_INFO: `/v3/projects/2d0c96c8150947c58293e0a3cc1f5ab5/users/1a7eee434bcc4d3894fbfc496d267ddc/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:07.074659 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50328979-e9fb-4cfa-8888-b20489911b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d0c96c8150947c58293e0a3cc1f5ab5, user_id=1a7eee434bcc4d3894fbfc496d267ddc, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:07.077550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50328979-e9fb-4cfa-8888-b20489911b37 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:07.147539 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 900/1789] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:06 2022] POST /identity/v3/users => generated 287 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:07.148937 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377047c6-c0cf-4046-bfd7-5349d2d8da3f admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:07.163503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377047c6-c0cf-4046-bfd7-5349d2d8da3f admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:07.169293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310853cc-6acb-4b54-b51e-ec1b7075db23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:07.170260 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310853cc-6acb-4b54-b51e-ec1b7075db23 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:07.170404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310853cc-6acb-4b54-b51e-ec1b7075db23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:07.170722 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310853cc-6acb-4b54-b51e-ec1b7075db23 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:07.171419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310853cc-6acb-4b54-b51e-ec1b7075db23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:07.173162 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310853cc-6acb-4b54-b51e-ec1b7075db23 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:07.201365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 890/1790] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:07 2022] PUT /identity/v3/projects/2d0c96c8150947c58293e0a3cc1f5ab5/users/1a7eee434bcc4d3894fbfc496d267ddc/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:07.202297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50328979-e9fb-4cfa-8888-b20489911b37 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:07.214348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 901/1791] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:07.221090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310853cc-6acb-4b54-b51e-ec1b7075db23 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:07.257677 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310853cc-6acb-4b54-b51e-ec1b7075db23 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:07.326861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50328979-e9fb-4cfa-8888-b20489911b37 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:07.334887 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d951fa3-dd27-42f5-9676-c93bf73e2c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:07.335633 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d951fa3-dd27-42f5-9676-c93bf73e2c4f admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:07.337368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d951fa3-dd27-42f5-9676-c93bf73e2c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:07.337662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d951fa3-dd27-42f5-9676-c93bf73e2c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:07.338338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d951fa3-dd27-42f5-9676-c93bf73e2c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:07.340800 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d951fa3-dd27-42f5-9676-c93bf73e2c4f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:07.358922 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 891/1792] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:07.360368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d951fa3-dd27-42f5-9676-c93bf73e2c4f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:07.361216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b626550-a65d-4dd5-848a-c5a319f870da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:07.363127 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b626550-a65d-4dd5-848a-c5a319f870da admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:07.363421 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b626550-a65d-4dd5-848a-c5a319f870da admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:07.363745 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b626550-a65d-4dd5-848a-c5a319f870da admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:07.364610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b626550-a65d-4dd5-848a-c5a319f870da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:07.367353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b626550-a65d-4dd5-848a-c5a319f870da admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:07.374195 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d951fa3-dd27-42f5-9676-c93bf73e2c4f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:07.382654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 902/1793] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:07.382654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b626550-a65d-4dd5-848a-c5a319f870da admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:07.382872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d04839-09e9-4541-b7fe-1ec8f85f3d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:07.383139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d04839-09e9-4541-b7fe-1ec8f85f3d2d admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:07.383442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d04839-09e9-4541-b7fe-1ec8f85f3d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:07.383722 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d04839-09e9-4541-b7fe-1ec8f85f3d2d admin admin] PATH_INFO: `/v3/projects/8a713c52671f40f4affea3a1f7fbae17/users/c9bec5015ffe47c6aaabef416c67c1d3/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:07.393041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b626550-a65d-4dd5-848a-c5a319f870da admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:07.398200 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc4f06-dd81-4ede-8cbe-054bc0cdce36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:07.398887 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc4f06-dd81-4ede-8cbe-054bc0cdce36 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:07.399246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc4f06-dd81-4ede-8cbe-054bc0cdce36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:07.419420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc4f06-dd81-4ede-8cbe-054bc0cdce36 admin admin] PATH_INFO: `/v3/projects/8fbe843aa127400fbaa5b26327f140c6/users/a2979e890ec44f42b3ccf80e2996dba9/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:07.419945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d04839-09e9-4541-b7fe-1ec8f85f3d2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a713c52671f40f4affea3a1f7fbae17, user_id=c9bec5015ffe47c6aaabef416c67c1d3, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:07.419945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d04839-09e9-4541-b7fe-1ec8f85f3d2d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:07.420095 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc4f06-dd81-4ede-8cbe-054bc0cdce36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fbe843aa127400fbaa5b26327f140c6, user_id=a2979e890ec44f42b3ccf80e2996dba9, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:07.420095 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc4f06-dd81-4ede-8cbe-054bc0cdce36 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:07.487505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 892/1794] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:07 2022] PUT /identity/v3/projects/8a713c52671f40f4affea3a1f7fbae17/users/c9bec5015ffe47c6aaabef416c67c1d3/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:07.488411 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d04839-09e9-4541-b7fe-1ec8f85f3d2d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:07.500249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 903/1795] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:07 2022] PUT /identity/v3/projects/8fbe843aa127400fbaa5b26327f140c6/users/a2979e890ec44f42b3ccf80e2996dba9/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:07.506486 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc4f06-dd81-4ede-8cbe-054bc0cdce36 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:07.671657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d04839-09e9-4541-b7fe-1ec8f85f3d2d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:07.679166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f49d96-4bd8-42a6-8f8f-273b849a7f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:07.680113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f49d96-4bd8-42a6-8f8f-273b849a7f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:07.680804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f49d96-4bd8-42a6-8f8f-273b849a7f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:07.681225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f49d96-4bd8-42a6-8f8f-273b849a7f83 admin admin] PATH_INFO: `/v3/projects/2d0c96c8150947c58293e0a3cc1f5ab5/users/1a7eee434bcc4d3894fbfc496d267ddc/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:07.681838 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc4f06-dd81-4ede-8cbe-054bc0cdce36 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:07.687055 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcd1d8-a3a5-4631-b929-a71219d27dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:07.687958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcd1d8-a3a5-4631-b929-a71219d27dcb admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:07.688458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcd1d8-a3a5-4631-b929-a71219d27dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:07.688801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcd1d8-a3a5-4631-b929-a71219d27dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:07.689647 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdcd1d8-a3a5-4631-b929-a71219d27dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:07.690357 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f49d96-4bd8-42a6-8f8f-273b849a7f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d0c96c8150947c58293e0a3cc1f5ab5, user_id=1a7eee434bcc4d3894fbfc496d267ddc, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:07.691900 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdcd1d8-a3a5-4631-b929-a71219d27dcb admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:07.694245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f49d96-4bd8-42a6-8f8f-273b849a7f83 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:07.703201 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 904/1796] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:07.704500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bcdd7-c675-4d4a-9487-42b22740f159 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:07.704790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bcdd7-c675-4d4a-9487-42b22740f159 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:07.705085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bcdd7-c675-4d4a-9487-42b22740f159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:07.712147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 893/1797] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:07 2022] PUT /identity/v3/projects/2d0c96c8150947c58293e0a3cc1f5ab5/users/1a7eee434bcc4d3894fbfc496d267ddc/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:07.713661 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a494fa-4564-45ec-9fee-77e632feff56 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:07.713951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a494fa-4564-45ec-9fee-77e632feff56 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:07.714221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a494fa-4564-45ec-9fee-77e632feff56 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:07.715883 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 894/1798] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 30 14:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:57:07.716733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a494fa-4564-45ec-9fee-77e632feff56 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:07.753218 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-898bcdd7-c675-4d4a-9487-42b22740f159 None None] MFA Rules not processed for user `c9bec5015ffe47c6aaabef416c67c1d3`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:07.754059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-898bcdd7-c675-4d4a-9487-42b22740f159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:07.774267 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:07.809544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a494fa-4564-45ec-9fee-77e632feff56 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:07.815274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4b2a6-8036-4c6e-b552-79bb6a0e7e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:07.815957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4b2a6-8036-4c6e-b552-79bb6a0e7e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:07.816383 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4b2a6-8036-4c6e-b552-79bb6a0e7e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:07.816493 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4b2a6-8036-4c6e-b552-79bb6a0e7e4c admin admin] PATH_INFO: `/v3/projects/8fbe843aa127400fbaa5b26327f140c6/users/a2979e890ec44f42b3ccf80e2996dba9/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:07.819432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4b2a6-8036-4c6e-b552-79bb6a0e7e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fbe843aa127400fbaa5b26327f140c6, user_id=a2979e890ec44f42b3ccf80e2996dba9, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:07.823722 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4b2a6-8036-4c6e-b552-79bb6a0e7e4c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:07.838402 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 895/1799] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:07 2022] PUT /identity/v3/projects/8fbe843aa127400fbaa5b26327f140c6/users/a2979e890ec44f42b3ccf80e2996dba9/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:07.841200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554a13e-c5d2-43f7-abbd-59c4bf2ec377 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:07.841200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554a13e-c5d2-43f7-abbd-59c4bf2ec377 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:07.841200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554a13e-c5d2-43f7-abbd-59c4bf2ec377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:07.848699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 905/1800] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:07.850836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53338560-85af-4b36-b007-dbafcdaa143f None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:07.851132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53338560-85af-4b36-b007-dbafcdaa143f None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:07.851405 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53338560-85af-4b36-b007-dbafcdaa143f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:07.870682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a554a13e-c5d2-43f7-abbd-59c4bf2ec377 None None] MFA Rules not processed for user `5b6e4c5bf2a54b28a55e5ae74b90ab4f`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:07.871945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a554a13e-c5d2-43f7-abbd-59c4bf2ec377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:07.885021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-53338560-85af-4b36-b007-dbafcdaa143f None None] MFA Rules not processed for user `1a7eee434bcc4d3894fbfc496d267ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:07.886237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-53338560-85af-4b36-b007-dbafcdaa143f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:07.998138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 906/1801] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:07.999798 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb11730-395e-4c88-b7fb-31f5cad44dc2 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:08.000510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb11730-395e-4c88-b7fb-31f5cad44dc2 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:08.000948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb11730-395e-4c88-b7fb-31f5cad44dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:08.038323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 896/1802] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 30 14:57:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:08.038996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554a13e-c5d2-43f7-abbd-59c4bf2ec377 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:08.046171 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-2cb11730-395e-4c88-b7fb-31f5cad44dc2 None None] MFA Rules not processed for user `a2979e890ec44f42b3ccf80e2996dba9`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:08.047610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-2cb11730-395e-4c88-b7fb-31f5cad44dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:08.147249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 907/1803] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:07 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:08.148690 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb11730-395e-4c88-b7fb-31f5cad44dc2 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:08.151327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554a13e-c5d2-43f7-abbd-59c4bf2ec377 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:08.153153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-2cb11730-395e-4c88-b7fb-31f5cad44dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:08.156411 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:08.165137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f543a52-f1dc-420a-ae0e-c99dd7e22b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:08.166057 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f543a52-f1dc-420a-ae0e-c99dd7e22b82 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:08.166252 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f543a52-f1dc-420a-ae0e-c99dd7e22b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:08.166614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f543a52-f1dc-420a-ae0e-c99dd7e22b82 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:08.167907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f543a52-f1dc-420a-ae0e-c99dd7e22b82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:08.170326 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f543a52-f1dc-420a-ae0e-c99dd7e22b82 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:08.268410 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 897/1804] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:08 2022] POST /identity/v3/projects => generated 377 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:08.269712 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f543a52-f1dc-420a-ae0e-c99dd7e22b82 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:08.274876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:08.277846 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb11730-395e-4c88-b7fb-31f5cad44dc2 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:08.378325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f543a52-f1dc-420a-ae0e-c99dd7e22b82 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:08.382717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610b9a4-f79f-4737-8ab4-bd7653d40ee1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:08.384689 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610b9a4-f79f-4737-8ab4-bd7653d40ee1 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:08.384976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610b9a4-f79f-4737-8ab4-bd7653d40ee1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:08.385252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610b9a4-f79f-4737-8ab4-bd7653d40ee1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:08.386845 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502850b-ffe8-4fc3-9b7f-26b5fb151b2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:08.387692 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502850b-ffe8-4fc3-9b7f-26b5fb151b2d service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:08.388362 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502850b-ffe8-4fc3-9b7f-26b5fb151b2d service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:08.389364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502850b-ffe8-4fc3-9b7f-26b5fb151b2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:08.405023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502850b-ffe8-4fc3-9b7f-26b5fb151b2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:08.406645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502850b-ffe8-4fc3-9b7f-26b5fb151b2d service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:08.431814 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 898/1805] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:08.434214 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502850b-ffe8-4fc3-9b7f-26b5fb151b2d service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:08.437194 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0502850b-ffe8-4fc3-9b7f-26b5fb151b2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:08.466419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502850b-ffe8-4fc3-9b7f-26b5fb151b2d service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:08.476967 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238caf58-7af8-4391-920f-1d3a68191f82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:08.476967 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238caf58-7af8-4391-920f-1d3a68191f82 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:08.478317 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238caf58-7af8-4391-920f-1d3a68191f82 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:08.478783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238caf58-7af8-4391-920f-1d3a68191f82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:08.487554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610b9a4-f79f-4737-8ab4-bd7653d40ee1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:08.492888 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610b9a4-f79f-4737-8ab4-bd7653d40ee1 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:08.521546 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 908/1806] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:08.522537 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610b9a4-f79f-4737-8ab4-bd7653d40ee1 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:08.576850 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238caf58-7af8-4391-920f-1d3a68191f82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:08.578317 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238caf58-7af8-4391-920f-1d3a68191f82 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:08.589774 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 899/1807] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:08 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:08.652116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610b9a4-f79f-4737-8ab4-bd7653d40ee1 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:08.657293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a83a1d-fa17-46e1-8bee-cc09132b6de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:08.657906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a83a1d-fa17-46e1-8bee-cc09132b6de3 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:08.658191 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a83a1d-fa17-46e1-8bee-cc09132b6de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:08.658515 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a83a1d-fa17-46e1-8bee-cc09132b6de3 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:08.659177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a83a1d-fa17-46e1-8bee-cc09132b6de3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:08.662822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a83a1d-fa17-46e1-8bee-cc09132b6de3 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:08.794879 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238caf58-7af8-4391-920f-1d3a68191f82 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:08.803413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 909/1808] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:08 2022] POST /identity/v3/users => generated 289 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:08.817011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238caf58-7af8-4391-920f-1d3a68191f82 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:08.832450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0f8dd-d31e-4b5e-b8a9-72323f6f4993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:08.833308 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0f8dd-d31e-4b5e-b8a9-72323f6f4993 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:08.833636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0f8dd-d31e-4b5e-b8a9-72323f6f4993 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:08.835164 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0f8dd-d31e-4b5e-b8a9-72323f6f4993 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:08.839523 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f0f8dd-d31e-4b5e-b8a9-72323f6f4993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:08.846765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f0f8dd-d31e-4b5e-b8a9-72323f6f4993 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:08.846891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:08.869111 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 900/1809] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:08.873127 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a83a1d-fa17-46e1-8bee-cc09132b6de3 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:08.885542 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a83a1d-fa17-46e1-8bee-cc09132b6de3 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:08.892948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a55564-f241-47f6-9b08-4f0e943b7d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:08.893830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a55564-f241-47f6-9b08-4f0e943b7d7f admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:08.894797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a55564-f241-47f6-9b08-4f0e943b7d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:08.895261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a55564-f241-47f6-9b08-4f0e943b7d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:08.896770 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a55564-f241-47f6-9b08-4f0e943b7d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:08.898605 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a55564-f241-47f6-9b08-4f0e943b7d7f admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:08.909891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:08.912930 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 910/1810] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:08.917252 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0f8dd-d31e-4b5e-b8a9-72323f6f4993 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:08.928629 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0f8dd-d31e-4b5e-b8a9-72323f6f4993 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:08.933404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038ba84-bc1e-4628-8e17-0a329c1c6b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:08.934059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038ba84-bc1e-4628-8e17-0a329c1c6b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:08.934340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038ba84-bc1e-4628-8e17-0a329c1c6b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:08.934616 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038ba84-bc1e-4628-8e17-0a329c1c6b7f admin admin] PATH_INFO: `/v3/projects/0ff0e37c1130469a85ec57c33db36389/users/e3bc1cc793ca4ec4a1e67441fe60c392/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:08.956455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038ba84-bc1e-4628-8e17-0a329c1c6b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ff0e37c1130469a85ec57c33db36389, user_id=e3bc1cc793ca4ec4a1e67441fe60c392, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:08.962540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038ba84-bc1e-4628-8e17-0a329c1c6b7f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:09.111251 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 901/1811] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:08 2022] PUT /identity/v3/projects/0ff0e37c1130469a85ec57c33db36389/users/e3bc1cc793ca4ec4a1e67441fe60c392/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:09.120812 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e33ab9-10f1-47a8-8d23-9fdd72064378 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:09.121226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e33ab9-10f1-47a8-8d23-9fdd72064378 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:09.121607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e33ab9-10f1-47a8-8d23-9fdd72064378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:09.205028 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-72e33ab9-10f1-47a8-8d23-9fdd72064378 None None] MFA Rules not processed for user `e3bc1cc793ca4ec4a1e67441fe60c392`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:09.206311 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-72e33ab9-10f1-47a8-8d23-9fdd72064378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:09.266404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:09.321450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 911/1812] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:09.330489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038ba84-bc1e-4628-8e17-0a329c1c6b7f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:09.451919 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038ba84-bc1e-4628-8e17-0a329c1c6b7f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:09.457912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f684b-de7f-4eb4-b7ee-771bc790af51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:09.459287 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f684b-de7f-4eb4-b7ee-771bc790af51 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:09.459651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f684b-de7f-4eb4-b7ee-771bc790af51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:09.459932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f684b-de7f-4eb4-b7ee-771bc790af51 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:09.460725 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f684b-de7f-4eb4-b7ee-771bc790af51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:09.463437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f684b-de7f-4eb4-b7ee-771bc790af51 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:09.518242 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:09.597902 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 902/1813] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:09 2022] POST /identity/v3/projects => generated 379 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:09.601533 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e33ab9-10f1-47a8-8d23-9fdd72064378 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:09.615629 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e33ab9-10f1-47a8-8d23-9fdd72064378 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:09.682957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c87ff5-3699-4292-a216-c2c808850d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:09.683646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c87ff5-3699-4292-a216-c2c808850d1c admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:09.683927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c87ff5-3699-4292-a216-c2c808850d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:09.684197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c87ff5-3699-4292-a216-c2c808850d1c admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:09.684844 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c87ff5-3699-4292-a216-c2c808850d1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:09.686864 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c87ff5-3699-4292-a216-c2c808850d1c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:09.773787 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 912/1814] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:09 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:09.779513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f684b-de7f-4eb4-b7ee-771bc790af51 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:09.792934 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f684b-de7f-4eb4-b7ee-771bc790af51 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:09.798035 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f686b-46c3-406d-9262-c382af165b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:09.798809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f686b-46c3-406d-9262-c382af165b04 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:09.799187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f686b-46c3-406d-9262-c382af165b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:09.800030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f686b-46c3-406d-9262-c382af165b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:09.800753 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f686b-46c3-406d-9262-c382af165b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:09.803305 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f686b-46c3-406d-9262-c382af165b04 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:09.816170 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 903/1815] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:09.821152 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c87ff5-3699-4292-a216-c2c808850d1c admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:09.833971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c87ff5-3699-4292-a216-c2c808850d1c admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:09.840680 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366b388-339b-4824-9d59-b2b44c64980b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:09.841365 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366b388-339b-4824-9d59-b2b44c64980b admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:09.841835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366b388-339b-4824-9d59-b2b44c64980b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:09.842125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366b388-339b-4824-9d59-b2b44c64980b admin admin] PATH_INFO: `/v3/projects/72524bff705546c59807dd510cc06b82/users/1039f69cab2247649fe01d42ad693308/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:09.867422 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366b388-339b-4824-9d59-b2b44c64980b admin admin] RBAC: Authorizing `identity:create_grant(project_id=72524bff705546c59807dd510cc06b82, user_id=1039f69cab2247649fe01d42ad693308, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:09.870835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366b388-339b-4824-9d59-b2b44c64980b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:09.881111 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f686b-46c3-406d-9262-c382af165b04 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:09.895435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f686b-46c3-406d-9262-c382af165b04 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:09.929933 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fd0a3-f105-4712-b539-5fc776b08175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:09.930671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fd0a3-f105-4712-b539-5fc776b08175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:09.930951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fd0a3-f105-4712-b539-5fc776b08175 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:09.931254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fd0a3-f105-4712-b539-5fc776b08175 admin admin] PATH_INFO: `/v3/users/0baa7e10632848b39a55f7e69abfe603` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:09.932558 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5fd0a3-f105-4712-b539-5fc776b08175 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0baa7e10632848b39a55f7e69abfe603)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:09.934680 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5fd0a3-f105-4712-b539-5fc776b08175 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:10.016547 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 913/1816] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:09 2022] PUT /identity/v3/projects/72524bff705546c59807dd510cc06b82/users/1039f69cab2247649fe01d42ad693308/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:10.020447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366b388-339b-4824-9d59-b2b44c64980b admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:10.061352 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:10.105457 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366b388-339b-4824-9d59-b2b44c64980b admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:10.111192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f5a95-8811-41b1-a5cd-66da9dcc2e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:10.111812 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f5a95-8811-41b1-a5cd-66da9dcc2e52 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:10.112109 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f5a95-8811-41b1-a5cd-66da9dcc2e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:10.112359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f5a95-8811-41b1-a5cd-66da9dcc2e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:10.113009 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f5a95-8811-41b1-a5cd-66da9dcc2e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:10.114655 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f5a95-8811-41b1-a5cd-66da9dcc2e52 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:10.131285 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 914/1817] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:10.133825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f5a95-8811-41b1-a5cd-66da9dcc2e52 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:10.145989 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:10.149249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f5a95-8811-41b1-a5cd-66da9dcc2e52 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:10.152633 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a5a56-0646-4d6d-a310-c08acd90f37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:10.153347 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a5a56-0646-4d6d-a310-c08acd90f37a admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:10.153629 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a5a56-0646-4d6d-a310-c08acd90f37a admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:10.153899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a5a56-0646-4d6d-a310-c08acd90f37a admin admin] PATH_INFO: `/v3/projects/72524bff705546c59807dd510cc06b82/users/1039f69cab2247649fe01d42ad693308/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:10.156467 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a5a56-0646-4d6d-a310-c08acd90f37a admin admin] RBAC: Authorizing `identity:create_grant(project_id=72524bff705546c59807dd510cc06b82, user_id=1039f69cab2247649fe01d42ad693308, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:10.159495 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a5a56-0646-4d6d-a310-c08acd90f37a admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:10.170843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1c5fd0a3-f105-4712-b539-5fc776b08175 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0baa7e10632848b39a55f7e69abfe603'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:10.183321 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 915/1818] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:10 2022] PUT /identity/v3/projects/72524bff705546c59807dd510cc06b82/users/1039f69cab2247649fe01d42ad693308/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:10.183510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1c5fd0a3-f105-4712-b539-5fc776b08175 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0baa7e10632848b39a55f7e69abfe603'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:10.183510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1c5fd0a3-f105-4712-b539-5fc776b08175 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0baa7e10632848b39a55f7e69abfe603'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:10.186114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5b91f-dec1-46ff-802c-a3c4731144f4 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:10.186442 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5b91f-dec1-46ff-802c-a3c4731144f4 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:10.186629 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5b91f-dec1-46ff-802c-a3c4731144f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:10.221323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-6ea5b91f-dec1-46ff-802c-a3c4731144f4 None None] MFA Rules not processed for user `1039f69cab2247649fe01d42ad693308`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:10.222570 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6ea5b91f-dec1-46ff-802c-a3c4731144f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:10.306937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1c5fd0a3-f105-4712-b539-5fc776b08175 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0baa7e10632848b39a55f7e69abfe603'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:10.327365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-1c5fd0a3-f105-4712-b539-5fc776b08175 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0baa7e10632848b39a55f7e69abfe603'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:10.334949 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 916/1819] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:10.341650 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 904/1820] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:09 2022] DELETE /identity/v3/users/0baa7e10632848b39a55f7e69abfe603 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:10.344242 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5b91f-dec1-46ff-802c-a3c4731144f4 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:10.346004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6ea5b91f-dec1-46ff-802c-a3c4731144f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:10.346609 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29c16e-a1f6-41d3-b4b9-56c6b6306e07 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:10.347248 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29c16e-a1f6-41d3-b4b9-56c6b6306e07 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:10.348028 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29c16e-a1f6-41d3-b4b9-56c6b6306e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:10.371722 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-fb29c16e-a1f6-41d3-b4b9-56c6b6306e07 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:10.373465 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fb29c16e-a1f6-41d3-b4b9-56c6b6306e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:10.445845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5b91f-dec1-46ff-802c-a3c4731144f4 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:10.448148 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 905/1821] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:10.483988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29c16e-a1f6-41d3-b4b9-56c6b6306e07 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:10.485801 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fb29c16e-a1f6-41d3-b4b9-56c6b6306e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:10.506262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8127605-2603-438b-8675-d2cc30cb1a78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:10.506982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8127605-2603-438b-8675-d2cc30cb1a78 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:10.507362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8127605-2603-438b-8675-d2cc30cb1a78 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:10.507640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8127605-2603-438b-8675-d2cc30cb1a78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:10.555094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29c16e-a1f6-41d3-b4b9-56c6b6306e07 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:10.559566 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bed4c4-a821-4098-b003-89c223cd1fad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:10.560355 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bed4c4-a821-4098-b003-89c223cd1fad service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:10.560785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bed4c4-a821-4098-b003-89c223cd1fad service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:10.561184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bed4c4-a821-4098-b003-89c223cd1fad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:10.573202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bed4c4-a821-4098-b003-89c223cd1fad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:10.575884 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bed4c4-a821-4098-b003-89c223cd1fad service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:10.588386 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 906/1822] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:10.614323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8127605-2603-438b-8675-d2cc30cb1a78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:10.616170 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8127605-2603-438b-8675-d2cc30cb1a78 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:10.639808 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 917/1823] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:10 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:10.718298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:10.776412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bed4c4-a821-4098-b003-89c223cd1fad service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:10.789563 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bed4c4-a821-4098-b003-89c223cd1fad service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:10.794029 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b002c53-fcdf-4748-80a7-bda2e6cf775f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:10.794832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b002c53-fcdf-4748-80a7-bda2e6cf775f service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:10.795248 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b002c53-fcdf-4748-80a7-bda2e6cf775f service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:10.795737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b002c53-fcdf-4748-80a7-bda2e6cf775f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:10.797805 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2b002c53-fcdf-4748-80a7-bda2e6cf775f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:10.864443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b002c53-fcdf-4748-80a7-bda2e6cf775f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:10.866366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b002c53-fcdf-4748-80a7-bda2e6cf775f service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:10.877738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 907/1824] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:11.343990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8127605-2603-438b-8675-d2cc30cb1a78 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:11.347360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-f8127605-2603-438b-8675-d2cc30cb1a78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:11.372280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8127605-2603-438b-8675-d2cc30cb1a78 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:11.385662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f8b08-7e19-4203-83b2-c8eb95479a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:11.393188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f8b08-7e19-4203-83b2-c8eb95479a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:11.393188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f8b08-7e19-4203-83b2-c8eb95479a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:11.393188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f8b08-7e19-4203-83b2-c8eb95479a79 admin admin] PATH_INFO: `/v3/projects/76b746e35f9c4268bb4c2bdfc9c02ee6` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:11.393188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f8b08-7e19-4203-83b2-c8eb95479a79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76b746e35f9c4268bb4c2bdfc9c02ee6)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:11.393188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f8b08-7e19-4203-83b2-c8eb95479a79 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:11.508152 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:11.651396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ef9f8b08-7e19-4203-83b2-c8eb95479a79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76b746e35f9c4268bb4c2bdfc9c02ee6', 'request_id': 'req-ef9f8b08-7e19-4203-83b2-c8eb95479a79'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:11.660070 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ef9f8b08-7e19-4203-83b2-c8eb95479a79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76b746e35f9c4268bb4c2bdfc9c02ee6', 'request_id': 'req-ef9f8b08-7e19-4203-83b2-c8eb95479a79'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:11.749919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:11.763309 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ef9f8b08-7e19-4203-83b2-c8eb95479a79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76b746e35f9c4268bb4c2bdfc9c02ee6', 'request_id': 'req-ef9f8b08-7e19-4203-83b2-c8eb95479a79'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:11.782350 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ef9f8b08-7e19-4203-83b2-c8eb95479a79 admin admin] The token cache is being invalidate because project 76b746e35f9c4268bb4c2bdfc9c02ee6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:11.782881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ef9f8b08-7e19-4203-83b2-c8eb95479a79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:11.784387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ef9f8b08-7e19-4203-83b2-c8eb95479a79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:11.788397 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 918/1825] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:11 2022] DELETE /identity/v3/projects/76b746e35f9c4268bb4c2bdfc9c02ee6 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:11.793999 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b002c53-fcdf-4748-80a7-bda2e6cf775f service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:11.810961 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b002c53-fcdf-4748-80a7-bda2e6cf775f service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:11.883477 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f3090-9a4c-448e-8908-3c0aad36b7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:11.884162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f3090-9a4c-448e-8908-3c0aad36b7b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:11.884393 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f3090-9a4c-448e-8908-3c0aad36b7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:11.884668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f3090-9a4c-448e-8908-3c0aad36b7b0 admin admin] PATH_INFO: `/v3/users/a2979e890ec44f42b3ccf80e2996dba9` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:11.886131 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f3090-9a4c-448e-8908-3c0aad36b7b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2979e890ec44f42b3ccf80e2996dba9)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:11.888361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f3090-9a4c-448e-8908-3c0aad36b7b0 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:12.117327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8b7f3090-9a4c-448e-8908-3c0aad36b7b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2979e890ec44f42b3ccf80e2996dba9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:12.124335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8b7f3090-9a4c-448e-8908-3c0aad36b7b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2979e890ec44f42b3ccf80e2996dba9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:12.125062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8b7f3090-9a4c-448e-8908-3c0aad36b7b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2979e890ec44f42b3ccf80e2996dba9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:12.203240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8b7f3090-9a4c-448e-8908-3c0aad36b7b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2979e890ec44f42b3ccf80e2996dba9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:12.216263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8b7f3090-9a4c-448e-8908-3c0aad36b7b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2979e890ec44f42b3ccf80e2996dba9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:12.222312 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 908/1826] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:11 2022] DELETE /identity/v3/users/a2979e890ec44f42b3ccf80e2996dba9 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:12.503336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:12.664408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f8b08-7e19-4203-83b2-c8eb95479a79 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:12.665801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ef9f8b08-7e19-4203-83b2-c8eb95479a79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:12.679247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f8b08-7e19-4203-83b2-c8eb95479a79 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:12.740333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:12.757691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a707b3-d520-4435-9926-c2a31ea7a43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:12.757691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a707b3-d520-4435-9926-c2a31ea7a43d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:12.757691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a707b3-d520-4435-9926-c2a31ea7a43d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:12.757691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a707b3-d520-4435-9926-c2a31ea7a43d admin admin] PATH_INFO: `/v3/projects/8fbe843aa127400fbaa5b26327f140c6` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:12.758007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a707b3-d520-4435-9926-c2a31ea7a43d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fbe843aa127400fbaa5b26327f140c6)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:12.758007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a707b3-d520-4435-9926-c2a31ea7a43d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:12.879524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-15a707b3-d520-4435-9926-c2a31ea7a43d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fbe843aa127400fbaa5b26327f140c6', 'request_id': 'req-15a707b3-d520-4435-9926-c2a31ea7a43d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:12.884904 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-15a707b3-d520-4435-9926-c2a31ea7a43d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fbe843aa127400fbaa5b26327f140c6', 'request_id': 'req-15a707b3-d520-4435-9926-c2a31ea7a43d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:13.000144 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-15a707b3-d520-4435-9926-c2a31ea7a43d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fbe843aa127400fbaa5b26327f140c6', 'request_id': 'req-15a707b3-d520-4435-9926-c2a31ea7a43d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:13.013020 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-15a707b3-d520-4435-9926-c2a31ea7a43d admin admin] The token cache is being invalidate because project 8fbe843aa127400fbaa5b26327f140c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:13.013225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-15a707b3-d520-4435-9926-c2a31ea7a43d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:13.014103 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-15a707b3-d520-4435-9926-c2a31ea7a43d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:13.015724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 919/1827] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:12 2022] DELETE /identity/v3/projects/8fbe843aa127400fbaa5b26327f140c6 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:13.020452 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7a579-4d78-4898-959e-ef14ce7e28e1 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:13.021448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7a579-4d78-4898-959e-ef14ce7e28e1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:13.021547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7a579-4d78-4898-959e-ef14ce7e28e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:13.041645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5ed7a579-4d78-4898-959e-ef14ce7e28e1 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:13.043191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5ed7a579-4d78-4898-959e-ef14ce7e28e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:13.115947 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 909/1828] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:13.128367 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6242279-c129-46af-b5c0-22152e70bf33 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:13.128652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6242279-c129-46af-b5c0-22152e70bf33 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:13.128933 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6242279-c129-46af-b5c0-22152e70bf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:13.151325 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-d6242279-c129-46af-b5c0-22152e70bf33 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:13.152547 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d6242279-c129-46af-b5c0-22152e70bf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:13.164025 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 920/1829] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:13.164993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7a579-4d78-4898-959e-ef14ce7e28e1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:13.172344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:13.176215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7a579-4d78-4898-959e-ef14ce7e28e1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:13.181355 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9e30c-cd56-44a3-86e7-3316f4056b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:13.182569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9e30c-cd56-44a3-86e7-3316f4056b6e admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:13.182569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9e30c-cd56-44a3-86e7-3316f4056b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:13.182569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9e30c-cd56-44a3-86e7-3316f4056b6e admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:13.183155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9e30c-cd56-44a3-86e7-3316f4056b6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:57:13.183457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9e30c-cd56-44a3-86e7-3316f4056b6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:13.185056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9e30c-cd56-44a3-86e7-3316f4056b6e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:13.197317 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 910/1830] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:57:13.201581 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6242279-c129-46af-b5c0-22152e70bf33 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:13.211377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6242279-c129-46af-b5c0-22152e70bf33 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:13.216301 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25d4fd-958c-4fba-8486-10e893b47c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:13.216908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25d4fd-958c-4fba-8486-10e893b47c2b admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:13.217188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25d4fd-958c-4fba-8486-10e893b47c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:13.217493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25d4fd-958c-4fba-8486-10e893b47c2b admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:13.218138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc25d4fd-958c-4fba-8486-10e893b47c2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:13.220120 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc25d4fd-958c-4fba-8486-10e893b47c2b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:13.313111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 921/1831] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:13 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:13.317300 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9e30c-cd56-44a3-86e7-3316f4056b6e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:13.327864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9e30c-cd56-44a3-86e7-3316f4056b6e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:13.385333 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cb47c-ad1a-4eff-8a27-cbac474f58cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:13.385937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cb47c-ad1a-4eff-8a27-cbac474f58cc admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:13.386216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cb47c-ad1a-4eff-8a27-cbac474f58cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:13.386488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cb47c-ad1a-4eff-8a27-cbac474f58cc admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:13.387166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736cb47c-ad1a-4eff-8a27-cbac474f58cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:13.389082 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736cb47c-ad1a-4eff-8a27-cbac474f58cc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:13.532534 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 911/1832] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:13 2022] POST /identity/v3/users => generated 294 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:13.533841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25d4fd-958c-4fba-8486-10e893b47c2b admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:13.544576 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25d4fd-958c-4fba-8486-10e893b47c2b admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:13.549198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f9915-6f19-4915-8ee7-894df2193dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:13.549803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f9915-6f19-4915-8ee7-894df2193dcf admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:13.550080 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f9915-6f19-4915-8ee7-894df2193dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:13.550356 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f9915-6f19-4915-8ee7-894df2193dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:13.551000 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f9915-6f19-4915-8ee7-894df2193dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:13.553171 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f9915-6f19-4915-8ee7-894df2193dcf admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:13.564313 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 922/1833] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:13.568742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cb47c-ad1a-4eff-8a27-cbac474f58cc admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:13.579001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cb47c-ad1a-4eff-8a27-cbac474f58cc admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:13.584752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed42332-99be-4cde-8470-2922d74a0ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:13.585349 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed42332-99be-4cde-8470-2922d74a0ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:13.585623 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed42332-99be-4cde-8470-2922d74a0ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:13.585920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed42332-99be-4cde-8470-2922d74a0ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:13.592128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed42332-99be-4cde-8470-2922d74a0ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:13.592128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed42332-99be-4cde-8470-2922d74a0ed6 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:13.598596 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 912/1834] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:57:13.602590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f9915-6f19-4915-8ee7-894df2193dcf admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:13.613065 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f9915-6f19-4915-8ee7-894df2193dcf admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:13.618707 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c181b52-6c2f-45fe-92e7-ccba24d3d750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:13.619415 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c181b52-6c2f-45fe-92e7-ccba24d3d750 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:13.619720 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c181b52-6c2f-45fe-92e7-ccba24d3d750 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:13.620058 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c181b52-6c2f-45fe-92e7-ccba24d3d750 admin admin] PATH_INFO: `/v3/projects/002ed56f94cd4b33ab6d8446526736fa/users/fc65f41935c54c0faa46c2ff7ce25618/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:13.635410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c181b52-6c2f-45fe-92e7-ccba24d3d750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=002ed56f94cd4b33ab6d8446526736fa, user_id=fc65f41935c54c0faa46c2ff7ce25618, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:13.638071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c181b52-6c2f-45fe-92e7-ccba24d3d750 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:13.743280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 923/1835] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:13 2022] PUT /identity/v3/projects/002ed56f94cd4b33ab6d8446526736fa/users/fc65f41935c54c0faa46c2ff7ce25618/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:13.753639 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a33465-c24c-40b0-80f4-ba181e36ec9f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:13.753918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a33465-c24c-40b0-80f4-ba181e36ec9f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:13.754187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a33465-c24c-40b0-80f4-ba181e36ec9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:13.782670 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e5a33465-c24c-40b0-80f4-ba181e36ec9f None None] MFA Rules not processed for user `fc65f41935c54c0faa46c2ff7ce25618`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:13.783856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e5a33465-c24c-40b0-80f4-ba181e36ec9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:13.847238 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 913/1836] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:13.852289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c181b52-6c2f-45fe-92e7-ccba24d3d750 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:13.915945 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c181b52-6c2f-45fe-92e7-ccba24d3d750 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:13.920987 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091abd32-60c1-47c2-9573-61f5ad62afa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:13.921627 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091abd32-60c1-47c2-9573-61f5ad62afa3 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:13.921900 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091abd32-60c1-47c2-9573-61f5ad62afa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:13.922164 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091abd32-60c1-47c2-9573-61f5ad62afa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:13.922827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091abd32-60c1-47c2-9573-61f5ad62afa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:13.925157 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091abd32-60c1-47c2-9573-61f5ad62afa3 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:13.925614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:14.012066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 924/1837] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:13 2022] POST /identity/v3/projects => generated 387 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:14.016208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a33465-c24c-40b0-80f4-ba181e36ec9f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:14.026914 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a33465-c24c-40b0-80f4-ba181e36ec9f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:14.082433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835925f-337b-4388-bcad-b4cfb5b25113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:14.082966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835925f-337b-4388-bcad-b4cfb5b25113 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:14.083286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835925f-337b-4388-bcad-b4cfb5b25113 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:14.083563 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835925f-337b-4388-bcad-b4cfb5b25113 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:14.084213 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835925f-337b-4388-bcad-b4cfb5b25113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:14.086140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835925f-337b-4388-bcad-b4cfb5b25113 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:14.217718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 914/1838] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:14 2022] POST /identity/v3/users => generated 300 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:14.219796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091abd32-60c1-47c2-9573-61f5ad62afa3 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:14.235823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091abd32-60c1-47c2-9573-61f5ad62afa3 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:14.241330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab0de2-900c-45d7-97ed-c69d35e9843f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:14.242155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab0de2-900c-45d7-97ed-c69d35e9843f admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:14.242436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab0de2-900c-45d7-97ed-c69d35e9843f admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:14.242706 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab0de2-900c-45d7-97ed-c69d35e9843f admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:14.243415 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab0de2-900c-45d7-97ed-c69d35e9843f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:14.245332 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab0de2-900c-45d7-97ed-c69d35e9843f admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:14.248403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:14.267852 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 925/1839] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:14.268711 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835925f-337b-4388-bcad-b4cfb5b25113 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:14.285938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835925f-337b-4388-bcad-b4cfb5b25113 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:14.302364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea74c40-93ec-4424-bdb4-f111e966de6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:14.302987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea74c40-93ec-4424-bdb4-f111e966de6a admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:14.304537 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea74c40-93ec-4424-bdb4-f111e966de6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:14.304537 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea74c40-93ec-4424-bdb4-f111e966de6a admin admin] PATH_INFO: `/v3/projects/b0287597ccd54afab1fe5831bdddb30c/users/7523a0dfdca4461483c7742b997315a2/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:14.323864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea74c40-93ec-4424-bdb4-f111e966de6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0287597ccd54afab1fe5831bdddb30c, user_id=7523a0dfdca4461483c7742b997315a2, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:14.337412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea74c40-93ec-4424-bdb4-f111e966de6a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:14.452880 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 915/1840] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:14 2022] PUT /identity/v3/projects/b0287597ccd54afab1fe5831bdddb30c/users/7523a0dfdca4461483c7742b997315a2/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:14.456897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab0de2-900c-45d7-97ed-c69d35e9843f admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:14.467921 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab0de2-900c-45d7-97ed-c69d35e9843f admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:14.531984 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305c721-4ac6-47cf-bc4f-5332f77cb612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:14.532624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305c721-4ac6-47cf-bc4f-5332f77cb612 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:14.532903 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305c721-4ac6-47cf-bc4f-5332f77cb612 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:14.533207 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305c721-4ac6-47cf-bc4f-5332f77cb612 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:14.533964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305c721-4ac6-47cf-bc4f-5332f77cb612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:14.535634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305c721-4ac6-47cf-bc4f-5332f77cb612 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:14.547011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 926/1841] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:14.551248 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea74c40-93ec-4424-bdb4-f111e966de6a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:14.561680 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea74c40-93ec-4424-bdb4-f111e966de6a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:14.573709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e51bc-7a71-4dc9-ab47-89f5e07da442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:14.574416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e51bc-7a71-4dc9-ab47-89f5e07da442 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:14.574698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e51bc-7a71-4dc9-ab47-89f5e07da442 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:14.574967 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e51bc-7a71-4dc9-ab47-89f5e07da442 admin admin] PATH_INFO: `/v3/projects/b0287597ccd54afab1fe5831bdddb30c/users/7523a0dfdca4461483c7742b997315a2/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:14.577381 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e51bc-7a71-4dc9-ab47-89f5e07da442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0287597ccd54afab1fe5831bdddb30c, user_id=7523a0dfdca4461483c7742b997315a2, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:14.580778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e51bc-7a71-4dc9-ab47-89f5e07da442 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:14.603017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 916/1842] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:14 2022] PUT /identity/v3/projects/b0287597ccd54afab1fe5831bdddb30c/users/7523a0dfdca4461483c7742b997315a2/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:14.614138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493f7a9-0d8c-4645-9aad-7a4c8fdfae71 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:14.615863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493f7a9-0d8c-4645-9aad-7a4c8fdfae71 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:14.616146 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493f7a9-0d8c-4645-9aad-7a4c8fdfae71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:14.690892 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-6493f7a9-0d8c-4645-9aad-7a4c8fdfae71 None None] MFA Rules not processed for user `7523a0dfdca4461483c7742b997315a2`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:14.694711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6493f7a9-0d8c-4645-9aad-7a4c8fdfae71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:14.843299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 927/1843] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:14 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:14.845257 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:14.853819 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e51bc-7a71-4dc9-ab47-89f5e07da442 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:14.854557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ed2e51bc-7a71-4dc9-ab47-89f5e07da442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:14.922156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e51bc-7a71-4dc9-ab47-89f5e07da442 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:14.926909 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af419a02-6c81-40f5-9597-a06eec422ef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:14.927787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af419a02-6c81-40f5-9597-a06eec422ef8 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:14.928651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af419a02-6c81-40f5-9597-a06eec422ef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:14.928997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af419a02-6c81-40f5-9597-a06eec422ef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:14.940665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af419a02-6c81-40f5-9597-a06eec422ef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:14.942237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af419a02-6c81-40f5-9597-a06eec422ef8 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:14.958289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 917/1844] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:15.057960 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493f7a9-0d8c-4645-9aad-7a4c8fdfae71 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:15.154856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493f7a9-0d8c-4645-9aad-7a4c8fdfae71 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:15.161969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7767b-518a-4119-bd9c-d8ca4b9cfcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:15.165341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7767b-518a-4119-bd9c-d8ca4b9cfcca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:15.165341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7767b-518a-4119-bd9c-d8ca4b9cfcca admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:15.165341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7767b-518a-4119-bd9c-d8ca4b9cfcca admin admin] PATH_INFO: `/v3/users/fc65f41935c54c0faa46c2ff7ce25618` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:15.165341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7767b-518a-4119-bd9c-d8ca4b9cfcca admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc65f41935c54c0faa46c2ff7ce25618)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:15.166575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7767b-518a-4119-bd9c-d8ca4b9cfcca admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:15.192590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:15.337584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-20a7767b-518a-4119-bd9c-d8ca4b9cfcca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc65f41935c54c0faa46c2ff7ce25618'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:15.346299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-20a7767b-518a-4119-bd9c-d8ca4b9cfcca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc65f41935c54c0faa46c2ff7ce25618'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:15.363442 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-20a7767b-518a-4119-bd9c-d8ca4b9cfcca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc65f41935c54c0faa46c2ff7ce25618'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:15.364556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-20a7767b-518a-4119-bd9c-d8ca4b9cfcca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc65f41935c54c0faa46c2ff7ce25618'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:15.365411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-20a7767b-518a-4119-bd9c-d8ca4b9cfcca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc65f41935c54c0faa46c2ff7ce25618'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:15.431563 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af419a02-6c81-40f5-9597-a06eec422ef8 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:15.433227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-af419a02-6c81-40f5-9597-a06eec422ef8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:15.446599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af419a02-6c81-40f5-9597-a06eec422ef8 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:15.451280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a47aa-9230-4f91-a9e2-5642ca8da5c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:15.451906 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a47aa-9230-4f91-a9e2-5642ca8da5c8 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:15.459437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a47aa-9230-4f91-a9e2-5642ca8da5c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:15.459437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a47aa-9230-4f91-a9e2-5642ca8da5c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:15.459437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-de7a47aa-9230-4f91-a9e2-5642ca8da5c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:15.469827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 928/1845] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:15 2022] DELETE /identity/v3/users/fc65f41935c54c0faa46c2ff7ce25618 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:15.474290 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cee2b-8020-4459-8a71-c8ef2a70501f None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:15.474574 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cee2b-8020-4459-8a71-c8ef2a70501f None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:15.476337 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cee2b-8020-4459-8a71-c8ef2a70501f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:15.516659 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-da8cee2b-8020-4459-8a71-c8ef2a70501f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:15.518037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-da8cee2b-8020-4459-8a71-c8ef2a70501f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:15.601594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a47aa-9230-4f91-a9e2-5642ca8da5c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:15.603132 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a47aa-9230-4f91-a9e2-5642ca8da5c8 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:15.611262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 929/1846] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:15.614915 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 918/1847] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:15.617581 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cee2b-8020-4459-8a71-c8ef2a70501f None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:15.619729 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-da8cee2b-8020-4459-8a71-c8ef2a70501f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:15.745185 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cee2b-8020-4459-8a71-c8ef2a70501f None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:15.750601 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce09ab-b531-4667-943e-b87438924c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:15.751341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce09ab-b531-4667-943e-b87438924c7c service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:15.751902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce09ab-b531-4667-943e-b87438924c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:15.752530 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce09ab-b531-4667-943e-b87438924c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:15.775838 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce09ab-b531-4667-943e-b87438924c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:15.777037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce09ab-b531-4667-943e-b87438924c7c service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:15.794914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 930/1848] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:16.293114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a47aa-9230-4f91-a9e2-5642ca8da5c8 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:16.294710 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-de7a47aa-9230-4f91-a9e2-5642ca8da5c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:16.305911 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:16.311164 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a47aa-9230-4f91-a9e2-5642ca8da5c8 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:16.316197 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d21673-7aa8-4256-ace3-e3f04a67c91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:16.317069 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d21673-7aa8-4256-ace3-e3f04a67c91d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:16.317442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d21673-7aa8-4256-ace3-e3f04a67c91d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:16.317889 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d21673-7aa8-4256-ace3-e3f04a67c91d admin admin] PATH_INFO: `/v3/projects/002ed56f94cd4b33ab6d8446526736fa` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:16.319803 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d21673-7aa8-4256-ace3-e3f04a67c91d admin admin] RBAC: Authorizing `identity:delete_project(project_id=002ed56f94cd4b33ab6d8446526736fa)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:16.322478 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d21673-7aa8-4256-ace3-e3f04a67c91d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:16.472058 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e1d21673-7aa8-4256-ace3-e3f04a67c91d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '002ed56f94cd4b33ab6d8446526736fa', 'request_id': 'req-e1d21673-7aa8-4256-ace3-e3f04a67c91d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:16.478196 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e1d21673-7aa8-4256-ace3-e3f04a67c91d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '002ed56f94cd4b33ab6d8446526736fa', 'request_id': 'req-e1d21673-7aa8-4256-ace3-e3f04a67c91d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:16.485309 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e1d21673-7aa8-4256-ace3-e3f04a67c91d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '002ed56f94cd4b33ab6d8446526736fa', 'request_id': 'req-e1d21673-7aa8-4256-ace3-e3f04a67c91d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:16.600135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e1d21673-7aa8-4256-ace3-e3f04a67c91d admin admin] The token cache is being invalidate because project 002ed56f94cd4b33ab6d8446526736fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:16.600509 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e1d21673-7aa8-4256-ace3-e3f04a67c91d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:16.601231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e1d21673-7aa8-4256-ace3-e3f04a67c91d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:16.602888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 919/1849] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:16 2022] DELETE /identity/v3/projects/002ed56f94cd4b33ab6d8446526736fa => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:16.607266 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce09ab-b531-4667-943e-b87438924c7c service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:16.617776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce09ab-b531-4667-943e-b87438924c7c service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:16.652211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c150a5-7677-41de-99d9-2fac975839a0 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:16.652725 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c150a5-7677-41de-99d9-2fac975839a0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:16.653167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c150a5-7677-41de-99d9-2fac975839a0 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:16.654529 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 920/1850] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 30 14:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:57:16.664589 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:16.665237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f1d82-ddf5-4fce-af49-c1382e700933 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:16.665537 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f1d82-ddf5-4fce-af49-c1382e700933 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:16.665839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f1d82-ddf5-4fce-af49-c1382e700933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:16.681526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32574f-0baf-4022-802d-b702aff4609b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:16.682183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32574f-0baf-4022-802d-b702aff4609b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:16.682460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32574f-0baf-4022-802d-b702aff4609b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:16.682735 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32574f-0baf-4022-802d-b702aff4609b admin admin] PATH_INFO: `/v3/users/7523a0dfdca4461483c7742b997315a2` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:16.683912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32574f-0baf-4022-802d-b702aff4609b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7523a0dfdca4461483c7742b997315a2)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:16.685833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32574f-0baf-4022-802d-b702aff4609b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:16.687293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-7a8f1d82-ddf5-4fce-af49-c1382e700933 None None] MFA Rules not processed for user `5b6e4c5bf2a54b28a55e5ae74b90ab4f`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:16.688506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7a8f1d82-ddf5-4fce-af49-c1382e700933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:16.769451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 921/1851] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 30 14:57:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:16.784274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f1d82-ddf5-4fce-af49-c1382e700933 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:16.786071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7a8f1d82-ddf5-4fce-af49-c1382e700933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:16.802933 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f1d82-ddf5-4fce-af49-c1382e700933 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:16.809871 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7e387-4461-4eef-be82-9f23930dbbec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:16.810503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7e387-4461-4eef-be82-9f23930dbbec service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:16.810853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7e387-4461-4eef-be82-9f23930dbbec service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:16.811450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7e387-4461-4eef-be82-9f23930dbbec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:16.823353 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7e387-4461-4eef-be82-9f23930dbbec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:16.825120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7e387-4461-4eef-be82-9f23930dbbec service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:16.836092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 922/1852] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:16.887136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1b32574f-0baf-4022-802d-b702aff4609b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7523a0dfdca4461483c7742b997315a2'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:16.897233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1b32574f-0baf-4022-802d-b702aff4609b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7523a0dfdca4461483c7742b997315a2'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:16.938176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1b32574f-0baf-4022-802d-b702aff4609b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7523a0dfdca4461483c7742b997315a2'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:16.958923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1b32574f-0baf-4022-802d-b702aff4609b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7523a0dfdca4461483c7742b997315a2'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:16.958923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1b32574f-0baf-4022-802d-b702aff4609b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7523a0dfdca4461483c7742b997315a2'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:17.035677 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 931/1853] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:16 2022] DELETE /identity/v3/users/7523a0dfdca4461483c7742b997315a2 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:17.681339 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7e387-4461-4eef-be82-9f23930dbbec service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:17.682839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bbc7e387-4461-4eef-be82-9f23930dbbec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:17.698371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7e387-4461-4eef-be82-9f23930dbbec service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:17.754754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483f79e-9aa8-4944-be85-5ade0939f496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:17.755424 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483f79e-9aa8-4944-be85-5ade0939f496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:17.755703 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483f79e-9aa8-4944-be85-5ade0939f496 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:17.756468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483f79e-9aa8-4944-be85-5ade0939f496 admin admin] PATH_INFO: `/v3/projects/b0287597ccd54afab1fe5831bdddb30c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:17.757580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483f79e-9aa8-4944-be85-5ade0939f496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0287597ccd54afab1fe5831bdddb30c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:17.759679 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483f79e-9aa8-4944-be85-5ade0939f496 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:17.926169 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:18.177571 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c483f79e-9aa8-4944-be85-5ade0939f496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0287597ccd54afab1fe5831bdddb30c', 'request_id': 'req-c483f79e-9aa8-4944-be85-5ade0939f496'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:18.184174 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c483f79e-9aa8-4944-be85-5ade0939f496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0287597ccd54afab1fe5831bdddb30c', 'request_id': 'req-c483f79e-9aa8-4944-be85-5ade0939f496'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:18.189819 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c483f79e-9aa8-4944-be85-5ade0939f496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0287597ccd54afab1fe5831bdddb30c', 'request_id': 'req-c483f79e-9aa8-4944-be85-5ade0939f496'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:19.146028 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c483f79e-9aa8-4944-be85-5ade0939f496 admin admin] The token cache is being invalidate because project b0287597ccd54afab1fe5831bdddb30c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:19.147307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c483f79e-9aa8-4944-be85-5ade0939f496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:19.148184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c483f79e-9aa8-4944-be85-5ade0939f496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:19.150155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 923/1854] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:17 2022] DELETE /identity/v3/projects/b0287597ccd54afab1fe5831bdddb30c => generated 0 bytes in 1469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:19.155779 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f26be1-bd2f-497c-9c8a-d7c079c15a14 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:19.156354 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f26be1-bd2f-497c-9c8a-d7c079c15a14 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:19.156354 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f26be1-bd2f-497c-9c8a-d7c079c15a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:19.178611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-72f26be1-bd2f-497c-9c8a-d7c079c15a14 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:19.180179 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-72f26be1-bd2f-497c-9c8a-d7c079c15a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:19.244215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 932/1855] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:19.252836 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572c0fb-91ed-49ef-b0eb-6ae3d9cc3f9e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:19.253253 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572c0fb-91ed-49ef-b0eb-6ae3d9cc3f9e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:19.253788 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572c0fb-91ed-49ef-b0eb-6ae3d9cc3f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:19.317666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5572c0fb-91ed-49ef-b0eb-6ae3d9cc3f9e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:19.319698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5572c0fb-91ed-49ef-b0eb-6ae3d9cc3f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:19.328101 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 924/1856] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:19.334209 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f26be1-bd2f-497c-9c8a-d7c079c15a14 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:19.344755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f26be1-bd2f-497c-9c8a-d7c079c15a14 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:19.349382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8b94e-8949-42c1-8ea5-2ac8507fe925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:19.351522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8b94e-8949-42c1-8ea5-2ac8507fe925 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:19.351522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8b94e-8949-42c1-8ea5-2ac8507fe925 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:19.351522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8b94e-8949-42c1-8ea5-2ac8507fe925 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:19.351522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8b94e-8949-42c1-8ea5-2ac8507fe925 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:57:19.351522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8b94e-8949-42c1-8ea5-2ac8507fe925 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:19.353233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac8b94e-8949-42c1-8ea5-2ac8507fe925 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:19.366598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 933/1857] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:57:19.371013 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572c0fb-91ed-49ef-b0eb-6ae3d9cc3f9e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:19.380733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572c0fb-91ed-49ef-b0eb-6ae3d9cc3f9e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:19.400139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57bf5be-338f-4321-921c-ca9e5464eca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:19.400139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bf5be-338f-4321-921c-ca9e5464eca5 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:19.400139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bf5be-338f-4321-921c-ca9e5464eca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:19.400139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bf5be-338f-4321-921c-ca9e5464eca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:19.400575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57bf5be-338f-4321-921c-ca9e5464eca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:19.400575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57bf5be-338f-4321-921c-ca9e5464eca5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:19.512662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 925/1858] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:19 2022] POST /identity/v3/projects => generated 367 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:19.514274 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8b94e-8949-42c1-8ea5-2ac8507fe925 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:19.524697 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8b94e-8949-42c1-8ea5-2ac8507fe925 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:19.575633 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:19.600166 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42579fa9-c2f5-4fd4-8946-598ace5604d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:19.601649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42579fa9-c2f5-4fd4-8946-598ace5604d0 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:19.601649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42579fa9-c2f5-4fd4-8946-598ace5604d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:19.601649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42579fa9-c2f5-4fd4-8946-598ace5604d0 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:19.602088 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42579fa9-c2f5-4fd4-8946-598ace5604d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:19.604304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42579fa9-c2f5-4fd4-8946-598ace5604d0 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:19.708265 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 934/1859] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:19 2022] POST /identity/v3/users => generated 284 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:19.714934 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57bf5be-338f-4321-921c-ca9e5464eca5 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:19.725353 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57bf5be-338f-4321-921c-ca9e5464eca5 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:19.730691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7e534-50fd-4f00-880b-8625c089e269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:19.731372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7e534-50fd-4f00-880b-8625c089e269 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:19.731648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7e534-50fd-4f00-880b-8625c089e269 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:19.731922 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7e534-50fd-4f00-880b-8625c089e269 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:19.732603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7e534-50fd-4f00-880b-8625c089e269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:19.734450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7e534-50fd-4f00-880b-8625c089e269 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:19.748637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 926/1860] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:19.750054 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42579fa9-c2f5-4fd4-8946-598ace5604d0 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:19.761149 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42579fa9-c2f5-4fd4-8946-598ace5604d0 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:19.765993 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713f2c3-fbed-4b2a-b10d-e7a9cb004706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:19.766673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713f2c3-fbed-4b2a-b10d-e7a9cb004706 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:19.766975 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713f2c3-fbed-4b2a-b10d-e7a9cb004706 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:19.767276 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713f2c3-fbed-4b2a-b10d-e7a9cb004706 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:19.767916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713f2c3-fbed-4b2a-b10d-e7a9cb004706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:19.769534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713f2c3-fbed-4b2a-b10d-e7a9cb004706 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:19.781017 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 935/1861] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:19.785430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7e534-50fd-4f00-880b-8625c089e269 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:19.796788 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7e534-50fd-4f00-880b-8625c089e269 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:19.801625 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbd97c-f834-4cab-b7b3-648e471ad98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:19.802831 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbd97c-f834-4cab-b7b3-648e471ad98e admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:19.802831 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbd97c-f834-4cab-b7b3-648e471ad98e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:19.803978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbd97c-f834-4cab-b7b3-648e471ad98e admin admin] PATH_INFO: `/v3/projects/39ac88123be14c39b30b55e87047ad22/users/0a537f6bb4f947b1bfe88bfe7d25787b/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:19.823666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dbd97c-f834-4cab-b7b3-648e471ad98e admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ac88123be14c39b30b55e87047ad22, user_id=0a537f6bb4f947b1bfe88bfe7d25787b, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:19.827808 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dbd97c-f834-4cab-b7b3-648e471ad98e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:19.921741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 927/1862] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:19 2022] PUT /identity/v3/projects/39ac88123be14c39b30b55e87047ad22/users/0a537f6bb4f947b1bfe88bfe7d25787b/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:19.934160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347863cd-8357-460d-ae94-9c69effc7752 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:19.934550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347863cd-8357-460d-ae94-9c69effc7752 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:19.934895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347863cd-8357-460d-ae94-9c69effc7752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:19.970090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-347863cd-8357-460d-ae94-9c69effc7752 None None] MFA Rules not processed for user `0a537f6bb4f947b1bfe88bfe7d25787b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:19.971766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-347863cd-8357-460d-ae94-9c69effc7752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:20.086487 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 936/1863] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:20.091034 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbd97c-f834-4cab-b7b3-648e471ad98e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:20.163749 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbd97c-f834-4cab-b7b3-648e471ad98e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:20.168425 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60f114-a0eb-4715-96d4-6af1e3f302d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:20.169133 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60f114-a0eb-4715-96d4-6af1e3f302d0 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:20.169411 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60f114-a0eb-4715-96d4-6af1e3f302d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:20.169679 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60f114-a0eb-4715-96d4-6af1e3f302d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:20.170368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60f114-a0eb-4715-96d4-6af1e3f302d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:20.172338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60f114-a0eb-4715-96d4-6af1e3f302d0 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:20.292798 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 928/1864] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:20 2022] POST /identity/v3/projects => generated 367 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:20.299320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347863cd-8357-460d-ae94-9c69effc7752 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:20.307605 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347863cd-8357-460d-ae94-9c69effc7752 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:20.367272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3fb91-dc85-47e2-b593-05c7011b06ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:20.367891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3fb91-dc85-47e2-b593-05c7011b06ba admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:20.368160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3fb91-dc85-47e2-b593-05c7011b06ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:20.368431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3fb91-dc85-47e2-b593-05c7011b06ba admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:20.369086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3fb91-dc85-47e2-b593-05c7011b06ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:20.371203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3fb91-dc85-47e2-b593-05c7011b06ba admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:20.563696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 937/1865] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:20 2022] POST /identity/v3/users => generated 290 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:20.568098 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60f114-a0eb-4715-96d4-6af1e3f302d0 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:20.580134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60f114-a0eb-4715-96d4-6af1e3f302d0 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:20.588584 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d6528-8a8f-4214-bd97-83f482d31f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:20.589277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d6528-8a8f-4214-bd97-83f482d31f6d admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:20.589628 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d6528-8a8f-4214-bd97-83f482d31f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:20.589964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d6528-8a8f-4214-bd97-83f482d31f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:20.590687 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d6528-8a8f-4214-bd97-83f482d31f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:20.592456 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d6528-8a8f-4214-bd97-83f482d31f6d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:20.607076 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 929/1866] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:20.608341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3fb91-dc85-47e2-b593-05c7011b06ba admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:20.618887 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3fb91-dc85-47e2-b593-05c7011b06ba admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:20.632816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd46a3-72d1-4b99-a87a-b82882ae7743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:20.633634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd46a3-72d1-4b99-a87a-b82882ae7743 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:20.634012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd46a3-72d1-4b99-a87a-b82882ae7743 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:20.634383 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd46a3-72d1-4b99-a87a-b82882ae7743 admin admin] PATH_INFO: `/v3/projects/040a3ee0971c47979d420fc505fd6a3d/users/008b4b5aea0a493192e41bcdeb5e99ba/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:20.670428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd46a3-72d1-4b99-a87a-b82882ae7743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=040a3ee0971c47979d420fc505fd6a3d, user_id=008b4b5aea0a493192e41bcdeb5e99ba, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:20.676092 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd46a3-72d1-4b99-a87a-b82882ae7743 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:20.760906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 938/1867] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:20 2022] PUT /identity/v3/projects/040a3ee0971c47979d420fc505fd6a3d/users/008b4b5aea0a493192e41bcdeb5e99ba/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:20.764496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d6528-8a8f-4214-bd97-83f482d31f6d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:20.778114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d6528-8a8f-4214-bd97-83f482d31f6d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:20.795050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:20.946496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff36b4-b3a4-4f68-a873-6b6b943d88de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:20.946496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff36b4-b3a4-4f68-a873-6b6b943d88de admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:20.948466 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff36b4-b3a4-4f68-a873-6b6b943d88de admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:20.963484 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff36b4-b3a4-4f68-a873-6b6b943d88de admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:20.971273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff36b4-b3a4-4f68-a873-6b6b943d88de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:20.972176 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff36b4-b3a4-4f68-a873-6b6b943d88de admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:20.987437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 930/1868] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:20.988341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd46a3-72d1-4b99-a87a-b82882ae7743 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:20.993103 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff36b4-b3a4-4f68-a873-6b6b943d88de admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:20.994869 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a0ff36b4-b3a4-4f68-a873-6b6b943d88de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:21.009666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff36b4-b3a4-4f68-a873-6b6b943d88de admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:21.015686 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713c68d-7889-48e1-b85a-3c65f637ffcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:21.015849 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd46a3-72d1-4b99-a87a-b82882ae7743 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:21.016268 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713c68d-7889-48e1-b85a-3c65f637ffcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:21.016544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713c68d-7889-48e1-b85a-3c65f637ffcb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:21.016821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713c68d-7889-48e1-b85a-3c65f637ffcb admin admin] PATH_INFO: `/v3/users/c9bec5015ffe47c6aaabef416c67c1d3` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:21.018434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713c68d-7889-48e1-b85a-3c65f637ffcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9bec5015ffe47c6aaabef416c67c1d3)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:21.020554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713c68d-7889-48e1-b85a-3c65f637ffcb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:21.023840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0a995-9671-41f9-8d18-792247d11a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:21.024730 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0a995-9671-41f9-8d18-792247d11a12 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:21.025016 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0a995-9671-41f9-8d18-792247d11a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:21.025288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0a995-9671-41f9-8d18-792247d11a12 admin admin] PATH_INFO: `/v3/projects/040a3ee0971c47979d420fc505fd6a3d/users/008b4b5aea0a493192e41bcdeb5e99ba/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:21.027823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0a995-9671-41f9-8d18-792247d11a12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=040a3ee0971c47979d420fc505fd6a3d, user_id=008b4b5aea0a493192e41bcdeb5e99ba, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:21.030766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0a995-9671-41f9-8d18-792247d11a12 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:21.058956 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 939/1869] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:20 2022] PUT /identity/v3/projects/040a3ee0971c47979d420fc505fd6a3d/users/008b4b5aea0a493192e41bcdeb5e99ba/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:21.069556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789eb17-2859-405e-a2eb-841df2bc14f6 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:21.069876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789eb17-2859-405e-a2eb-841df2bc14f6 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:21.070143 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789eb17-2859-405e-a2eb-841df2bc14f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:21.142985 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:21.163743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-1789eb17-2859-405e-a2eb-841df2bc14f6 None None] MFA Rules not processed for user `008b4b5aea0a493192e41bcdeb5e99ba`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:21.165220 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1789eb17-2859-405e-a2eb-841df2bc14f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:21.274831 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e713c68d-7889-48e1-b85a-3c65f637ffcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9bec5015ffe47c6aaabef416c67c1d3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:21.292502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e713c68d-7889-48e1-b85a-3c65f637ffcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9bec5015ffe47c6aaabef416c67c1d3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:21.296073 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e713c68d-7889-48e1-b85a-3c65f637ffcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9bec5015ffe47c6aaabef416c67c1d3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:21.335529 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 940/1870] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:21.347066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789eb17-2859-405e-a2eb-841df2bc14f6 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:21.349109 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1789eb17-2859-405e-a2eb-841df2bc14f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:21.420613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e713c68d-7889-48e1-b85a-3c65f637ffcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9bec5015ffe47c6aaabef416c67c1d3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:21.428296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789eb17-2859-405e-a2eb-841df2bc14f6 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:21.437001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e713c68d-7889-48e1-b85a-3c65f637ffcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9bec5015ffe47c6aaabef416c67c1d3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:21.437174 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd9eae-03a1-4445-8ae6-21651edaf11a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:21.437790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd9eae-03a1-4445-8ae6-21651edaf11a service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:21.438093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd9eae-03a1-4445-8ae6-21651edaf11a service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:21.438418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd9eae-03a1-4445-8ae6-21651edaf11a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:21.448411 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 931/1871] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:20 2022] DELETE /identity/v3/users/c9bec5015ffe47c6aaabef416c67c1d3 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:21.451440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd9eae-03a1-4445-8ae6-21651edaf11a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:21.456162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07b097-a286-4b54-87bf-9e391bc01a5f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:21.456162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07b097-a286-4b54-87bf-9e391bc01a5f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:21.456162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07b097-a286-4b54-87bf-9e391bc01a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:21.456498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd9eae-03a1-4445-8ae6-21651edaf11a service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:21.464859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 941/1872] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:21.477701 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1d07b097-a286-4b54-87bf-9e391bc01a5f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:21.485355 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1d07b097-a286-4b54-87bf-9e391bc01a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:21.557025 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 932/1873] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:21.571255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd9eae-03a1-4445-8ae6-21651edaf11a service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:21.572674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c1fd9eae-03a1-4445-8ae6-21651edaf11a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:21.643970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd9eae-03a1-4445-8ae6-21651edaf11a service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:21.694501 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869df97e-55b6-40f6-a7de-51d7c42bcebe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:21.695181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869df97e-55b6-40f6-a7de-51d7c42bcebe service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:21.695464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869df97e-55b6-40f6-a7de-51d7c42bcebe service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:21.695735 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869df97e-55b6-40f6-a7de-51d7c42bcebe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:21.702692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:21.704905 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869df97e-55b6-40f6-a7de-51d7c42bcebe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:21.706323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869df97e-55b6-40f6-a7de-51d7c42bcebe service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:21.725939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 942/1874] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:57:21.733583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07b097-a286-4b54-87bf-9e391bc01a5f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:21.767248 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:21.848135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07b097-a286-4b54-87bf-9e391bc01a5f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:21.853251 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f62f1-cb5c-4ef0-9868-06a768806fdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:21.853863 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f62f1-cb5c-4ef0-9868-06a768806fdc service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:21.854145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f62f1-cb5c-4ef0-9868-06a768806fdc service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:21.854415 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f62f1-cb5c-4ef0-9868-06a768806fdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:21.856430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-518f62f1-cb5c-4ef0-9868-06a768806fdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:21.929977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f62f1-cb5c-4ef0-9868-06a768806fdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:21.931401 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f62f1-cb5c-4ef0-9868-06a768806fdc service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:21.947528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 933/1875] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:22.442458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869df97e-55b6-40f6-a7de-51d7c42bcebe service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:22.445249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-869df97e-55b6-40f6-a7de-51d7c42bcebe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:22.460091 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869df97e-55b6-40f6-a7de-51d7c42bcebe service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:22.465526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db227a4-02e1-449f-a078-7d2160ff66ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:22.466179 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db227a4-02e1-449f-a078-7d2160ff66ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:22.466462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db227a4-02e1-449f-a078-7d2160ff66ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:22.466957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db227a4-02e1-449f-a078-7d2160ff66ba admin admin] PATH_INFO: `/v3/projects/8a713c52671f40f4affea3a1f7fbae17` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:22.468336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db227a4-02e1-449f-a078-7d2160ff66ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a713c52671f40f4affea3a1f7fbae17)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:22.470254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db227a4-02e1-449f-a078-7d2160ff66ba admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:22.660746 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-6db227a4-02e1-449f-a078-7d2160ff66ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a713c52671f40f4affea3a1f7fbae17', 'request_id': 'req-6db227a4-02e1-449f-a078-7d2160ff66ba'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:22.667022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-6db227a4-02e1-449f-a078-7d2160ff66ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a713c52671f40f4affea3a1f7fbae17', 'request_id': 'req-6db227a4-02e1-449f-a078-7d2160ff66ba'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:22.762628 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-6db227a4-02e1-449f-a078-7d2160ff66ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a713c52671f40f4affea3a1f7fbae17', 'request_id': 'req-6db227a4-02e1-449f-a078-7d2160ff66ba'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:22.774430 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-6db227a4-02e1-449f-a078-7d2160ff66ba admin admin] The token cache is being invalidate because project 8a713c52671f40f4affea3a1f7fbae17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:22.774725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-6db227a4-02e1-449f-a078-7d2160ff66ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:22.775421 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-6db227a4-02e1-449f-a078-7d2160ff66ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:22.776949 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 943/1876] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:22 2022] DELETE /identity/v3/projects/8a713c52671f40f4affea3a1f7fbae17 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:22.781049 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f62f1-cb5c-4ef0-9868-06a768806fdc service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:22.792536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f62f1-cb5c-4ef0-9868-06a768806fdc service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:22.861536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe759c80-b121-43e0-8041-3abb2a165cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:22.862180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe759c80-b121-43e0-8041-3abb2a165cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:22.862481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe759c80-b121-43e0-8041-3abb2a165cab admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:22.862721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe759c80-b121-43e0-8041-3abb2a165cab admin admin] PATH_INFO: `/v3/users/e3bc1cc793ca4ec4a1e67441fe60c392` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:22.863935 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe759c80-b121-43e0-8041-3abb2a165cab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3bc1cc793ca4ec4a1e67441fe60c392)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:22.866634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe759c80-b121-43e0-8041-3abb2a165cab admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:23.189888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-fe759c80-b121-43e0-8041-3abb2a165cab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3bc1cc793ca4ec4a1e67441fe60c392'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:23.200142 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-fe759c80-b121-43e0-8041-3abb2a165cab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3bc1cc793ca4ec4a1e67441fe60c392'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:23.201377 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-fe759c80-b121-43e0-8041-3abb2a165cab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3bc1cc793ca4ec4a1e67441fe60c392'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:23.362791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-fe759c80-b121-43e0-8041-3abb2a165cab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3bc1cc793ca4ec4a1e67441fe60c392'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:23.371182 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:23.376573 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-fe759c80-b121-43e0-8041-3abb2a165cab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3bc1cc793ca4ec4a1e67441fe60c392'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:23.383442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 934/1877] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:22 2022] DELETE /identity/v3/users/e3bc1cc793ca4ec4a1e67441fe60c392 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:23.985356 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db227a4-02e1-449f-a078-7d2160ff66ba admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:23.987880 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6db227a4-02e1-449f-a078-7d2160ff66ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:24.003968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db227a4-02e1-449f-a078-7d2160ff66ba admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:24.072036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:24.072725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:24.073003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:24.073275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f admin admin] PATH_INFO: `/v3/projects/0ff0e37c1130469a85ec57c33db36389` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:24.074459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ff0e37c1130469a85ec57c33db36389)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:24.076414 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:24.195472 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ff0e37c1130469a85ec57c33db36389', 'request_id': 'req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:24.200881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ff0e37c1130469a85ec57c33db36389', 'request_id': 'req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:24.302346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ff0e37c1130469a85ec57c33db36389', 'request_id': 'req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:24.313833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f admin admin] The token cache is being invalidate because project 0ff0e37c1130469a85ec57c33db36389 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:24.314114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:24.314736 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:24.316251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 944/1878] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:23 2022] DELETE /identity/v3/projects/0ff0e37c1130469a85ec57c33db36389 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:24.320653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe759c80-b121-43e0-8041-3abb2a165cab admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:24.322054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fe759c80-b121-43e0-8041-3abb2a165cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:24.335674 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe759c80-b121-43e0-8041-3abb2a165cab admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:24.391156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f3328-f359-4639-ae8c-c30fbdc86ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:24.391793 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f3328-f359-4639-ae8c-c30fbdc86ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:24.392072 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f3328-f359-4639-ae8c-c30fbdc86ece admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:24.392344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f3328-f359-4639-ae8c-c30fbdc86ece admin admin] PATH_INFO: `/v3/users/1039f69cab2247649fe01d42ad693308` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:24.393607 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f3328-f359-4639-ae8c-c30fbdc86ece admin admin] RBAC: Authorizing `identity:delete_user(user_id=1039f69cab2247649fe01d42ad693308)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:24.395611 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f3328-f359-4639-ae8c-c30fbdc86ece admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:24.604646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a87f3328-f359-4639-ae8c-c30fbdc86ece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1039f69cab2247649fe01d42ad693308'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:24.611539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a87f3328-f359-4639-ae8c-c30fbdc86ece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1039f69cab2247649fe01d42ad693308'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:24.612238 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a87f3328-f359-4639-ae8c-c30fbdc86ece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1039f69cab2247649fe01d42ad693308'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:24.673172 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a87f3328-f359-4639-ae8c-c30fbdc86ece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1039f69cab2247649fe01d42ad693308'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:24.685921 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a87f3328-f359-4639-ae8c-c30fbdc86ece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1039f69cab2247649fe01d42ad693308'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:24.691880 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 935/1879] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:24 2022] DELETE /identity/v3/users/1039f69cab2247649fe01d42ad693308 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:25.132868 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:25.134586 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:25.147834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2e04d-d55a-4559-89fd-6fcd2a46a74f admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:25.154107 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:25.205020 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fa743-df53-4504-88b3-f247f9285e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:25.205637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fa743-df53-4504-88b3-f247f9285e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:25.205919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fa743-df53-4504-88b3-f247f9285e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:25.206194 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fa743-df53-4504-88b3-f247f9285e44 admin admin] PATH_INFO: `/v3/projects/72524bff705546c59807dd510cc06b82` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:25.207513 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711fa743-df53-4504-88b3-f247f9285e44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72524bff705546c59807dd510cc06b82)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:25.209434 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711fa743-df53-4504-88b3-f247f9285e44 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:25.360516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-711fa743-df53-4504-88b3-f247f9285e44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72524bff705546c59807dd510cc06b82', 'request_id': 'req-711fa743-df53-4504-88b3-f247f9285e44'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:25.366644 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-711fa743-df53-4504-88b3-f247f9285e44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72524bff705546c59807dd510cc06b82', 'request_id': 'req-711fa743-df53-4504-88b3-f247f9285e44'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:25.444968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-711fa743-df53-4504-88b3-f247f9285e44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72524bff705546c59807dd510cc06b82', 'request_id': 'req-711fa743-df53-4504-88b3-f247f9285e44'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:25.456951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-711fa743-df53-4504-88b3-f247f9285e44 admin admin] The token cache is being invalidate because project 72524bff705546c59807dd510cc06b82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:25.461409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-711fa743-df53-4504-88b3-f247f9285e44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:25.461409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-711fa743-df53-4504-88b3-f247f9285e44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:25.461409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 945/1880] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:25 2022] DELETE /identity/v3/projects/72524bff705546c59807dd510cc06b82 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:25.465844 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b501b3d-52eb-471f-81b4-0a46236c0c5c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:25.466311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b501b3d-52eb-471f-81b4-0a46236c0c5c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:25.466439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b501b3d-52eb-471f-81b4-0a46236c0c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:25.488674 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3b501b3d-52eb-471f-81b4-0a46236c0c5c None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:25.495655 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3b501b3d-52eb-471f-81b4-0a46236c0c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:25.546499 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 936/1881] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:25.554903 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c67b7-8600-4a0b-9ba2-fa59b981d063 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:25.555244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c67b7-8600-4a0b-9ba2-fa59b981d063 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:25.555526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c67b7-8600-4a0b-9ba2-fa59b981d063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:25.575015 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-0d0c67b7-8600-4a0b-9ba2-fa59b981d063 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:25.576423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0d0c67b7-8600-4a0b-9ba2-fa59b981d063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:25.584918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 946/1882] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:25.589088 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b501b3d-52eb-471f-81b4-0a46236c0c5c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:25.600336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b501b3d-52eb-471f-81b4-0a46236c0c5c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:25.604859 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfea5f-f571-438a-b7aa-63244eea830a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:25.605450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfea5f-f571-438a-b7aa-63244eea830a admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:25.605729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfea5f-f571-438a-b7aa-63244eea830a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:25.605999 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfea5f-f571-438a-b7aa-63244eea830a admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:25.606640 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bfea5f-f571-438a-b7aa-63244eea830a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:57:25.606944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bfea5f-f571-438a-b7aa-63244eea830a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:25.608844 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bfea5f-f571-438a-b7aa-63244eea830a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:25.611117 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:25.620281 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 937/1883] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:57:25.624485 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c67b7-8600-4a0b-9ba2-fa59b981d063 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:25.633960 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c67b7-8600-4a0b-9ba2-fa59b981d063 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:25.638678 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f177d-c6c0-4f9b-81a4-5aa677749b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:25.639296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f177d-c6c0-4f9b-81a4-5aa677749b95 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:25.639580 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f177d-c6c0-4f9b-81a4-5aa677749b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:25.639854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f177d-c6c0-4f9b-81a4-5aa677749b95 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:25.640497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f177d-c6c0-4f9b-81a4-5aa677749b95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:25.642394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f177d-c6c0-4f9b-81a4-5aa677749b95 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:25.732855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 947/1884] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:25 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:25.737120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfea5f-f571-438a-b7aa-63244eea830a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:25.747589 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfea5f-f571-438a-b7aa-63244eea830a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:25.808003 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89559599-8675-408c-ae2c-05090f0f16e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:25.809203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89559599-8675-408c-ae2c-05090f0f16e6 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:25.809683 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89559599-8675-408c-ae2c-05090f0f16e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:25.810013 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89559599-8675-408c-ae2c-05090f0f16e6 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:25.812282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89559599-8675-408c-ae2c-05090f0f16e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:25.815311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89559599-8675-408c-ae2c-05090f0f16e6 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:25.924846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 938/1885] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:25 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:25.928905 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f177d-c6c0-4f9b-81a4-5aa677749b95 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:25.940554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f177d-c6c0-4f9b-81a4-5aa677749b95 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:25.945848 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fa007-315d-44aa-a647-53949301d25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:25.946546 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fa007-315d-44aa-a647-53949301d25d admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:25.946825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fa007-315d-44aa-a647-53949301d25d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:25.947117 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fa007-315d-44aa-a647-53949301d25d admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:25.947793 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fa007-315d-44aa-a647-53949301d25d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:25.949483 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fa007-315d-44aa-a647-53949301d25d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:25.959487 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 948/1886] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:25.964251 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89559599-8675-408c-ae2c-05090f0f16e6 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:25.974917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89559599-8675-408c-ae2c-05090f0f16e6 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:25.979576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e83e27-668d-468f-98bd-ee5752863736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:25.980180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e83e27-668d-468f-98bd-ee5752863736 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:25.980458 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e83e27-668d-468f-98bd-ee5752863736 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:25.980729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e83e27-668d-468f-98bd-ee5752863736 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:25.981372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e83e27-668d-468f-98bd-ee5752863736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:25.983476 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e83e27-668d-468f-98bd-ee5752863736 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:25.994468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 939/1887] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:25.998774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fa007-315d-44aa-a647-53949301d25d admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:26.009834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fa007-315d-44aa-a647-53949301d25d admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:26.016223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db882251-50fb-4a38-b455-1f742b03a9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:26.016889 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db882251-50fb-4a38-b455-1f742b03a9ba admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:26.017192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db882251-50fb-4a38-b455-1f742b03a9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:26.017601 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db882251-50fb-4a38-b455-1f742b03a9ba admin admin] PATH_INFO: `/v3/projects/924a1b6b958f4d86adc0a9ca294f59c0/users/404f7e5625764674afdb9d2828aa419c/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:26.034104 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db882251-50fb-4a38-b455-1f742b03a9ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=924a1b6b958f4d86adc0a9ca294f59c0, user_id=404f7e5625764674afdb9d2828aa419c, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:26.036875 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db882251-50fb-4a38-b455-1f742b03a9ba admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:26.144572 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 949/1888] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:25 2022] PUT /identity/v3/projects/924a1b6b958f4d86adc0a9ca294f59c0/users/404f7e5625764674afdb9d2828aa419c/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:26.157007 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f3fa0-2fbe-4470-829f-aec24a54c7af None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:26.157293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f3fa0-2fbe-4470-829f-aec24a54c7af None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:26.157571 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f3fa0-2fbe-4470-829f-aec24a54c7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:26.195511 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-220f3fa0-2fbe-4470-829f-aec24a54c7af None None] MFA Rules not processed for user `404f7e5625764674afdb9d2828aa419c`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:26.196718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-220f3fa0-2fbe-4470-829f-aec24a54c7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:26.291685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 940/1889] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:26.295668 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db882251-50fb-4a38-b455-1f742b03a9ba admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:26.341903 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:26.367970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db882251-50fb-4a38-b455-1f742b03a9ba admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:26.372950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecab910-3915-4f27-9c32-16a11d7c67a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:26.373600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecab910-3915-4f27-9c32-16a11d7c67a7 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:26.373879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecab910-3915-4f27-9c32-16a11d7c67a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:26.374197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecab910-3915-4f27-9c32-16a11d7c67a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:26.381167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecab910-3915-4f27-9c32-16a11d7c67a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:26.381167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecab910-3915-4f27-9c32-16a11d7c67a7 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:26.504546 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 950/1890] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:26 2022] POST /identity/v3/projects => generated 379 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:26.509120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f3fa0-2fbe-4470-829f-aec24a54c7af None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:26.525249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f3fa0-2fbe-4470-829f-aec24a54c7af None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:26.618327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9349f-aef1-421a-8924-fee04e37bed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:26.618999 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9349f-aef1-421a-8924-fee04e37bed7 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:26.619312 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9349f-aef1-421a-8924-fee04e37bed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:26.619590 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9349f-aef1-421a-8924-fee04e37bed7 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:26.620588 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9349f-aef1-421a-8924-fee04e37bed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:26.622362 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9349f-aef1-421a-8924-fee04e37bed7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:26.633095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:26.717477 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 941/1891] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:26 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:26.721538 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecab910-3915-4f27-9c32-16a11d7c67a7 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:26.734570 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecab910-3915-4f27-9c32-16a11d7c67a7 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:26.739262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868af2b2-4132-4500-a16e-33d2719ed248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:26.739873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868af2b2-4132-4500-a16e-33d2719ed248 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:26.740147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868af2b2-4132-4500-a16e-33d2719ed248 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:26.740586 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868af2b2-4132-4500-a16e-33d2719ed248 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:26.741869 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868af2b2-4132-4500-a16e-33d2719ed248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:26.743578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868af2b2-4132-4500-a16e-33d2719ed248 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:26.754510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 951/1892] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:26.758970 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9349f-aef1-421a-8924-fee04e37bed7 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:26.769056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9349f-aef1-421a-8924-fee04e37bed7 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:26.773587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332687bf-c8f9-49ee-9203-77363833042e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:26.774236 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332687bf-c8f9-49ee-9203-77363833042e admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:26.774511 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332687bf-c8f9-49ee-9203-77363833042e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:26.774780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332687bf-c8f9-49ee-9203-77363833042e admin admin] PATH_INFO: `/v3/projects/a9852c5fe67746a2b4e3fc36c015ddb4/users/a0b6875622fc4861a4568c3b3f2d75ca/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:26.791254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332687bf-c8f9-49ee-9203-77363833042e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9852c5fe67746a2b4e3fc36c015ddb4, user_id=a0b6875622fc4861a4568c3b3f2d75ca, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:26.794002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332687bf-c8f9-49ee-9203-77363833042e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:26.894150 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 942/1893] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:26 2022] PUT /identity/v3/projects/a9852c5fe67746a2b4e3fc36c015ddb4/users/a0b6875622fc4861a4568c3b3f2d75ca/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:26.898748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868af2b2-4132-4500-a16e-33d2719ed248 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:26.920956 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868af2b2-4132-4500-a16e-33d2719ed248 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:26.980859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28674e7b-7d10-4621-8076-94da728328d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:26.981479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28674e7b-7d10-4621-8076-94da728328d3 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:26.981756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28674e7b-7d10-4621-8076-94da728328d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:26.982025 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28674e7b-7d10-4621-8076-94da728328d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:26.983213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28674e7b-7d10-4621-8076-94da728328d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:26.984812 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28674e7b-7d10-4621-8076-94da728328d3 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:26.995968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 952/1894] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:26.999984 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332687bf-c8f9-49ee-9203-77363833042e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:27.010487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332687bf-c8f9-49ee-9203-77363833042e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:27.015065 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949440d-20d2-47c0-a78b-f2f29a1841eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:27.015739 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949440d-20d2-47c0-a78b-f2f29a1841eb admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:27.016019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949440d-20d2-47c0-a78b-f2f29a1841eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:27.016293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949440d-20d2-47c0-a78b-f2f29a1841eb admin admin] PATH_INFO: `/v3/projects/a9852c5fe67746a2b4e3fc36c015ddb4/users/a0b6875622fc4861a4568c3b3f2d75ca/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:27.018548 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949440d-20d2-47c0-a78b-f2f29a1841eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9852c5fe67746a2b4e3fc36c015ddb4, user_id=a0b6875622fc4861a4568c3b3f2d75ca, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:27.021786 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949440d-20d2-47c0-a78b-f2f29a1841eb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:27.033544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 943/1895] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:26 2022] PUT /identity/v3/projects/a9852c5fe67746a2b4e3fc36c015ddb4/users/a0b6875622fc4861a4568c3b3f2d75ca/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:27.045425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffb021-5f79-4f08-a9c2-026ba2cdb07c None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:27.045801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffb021-5f79-4f08-a9c2-026ba2cdb07c None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:27.046148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffb021-5f79-4f08-a9c2-026ba2cdb07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:27.079303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-8cffb021-5f79-4f08-a9c2-026ba2cdb07c None None] MFA Rules not processed for user `a0b6875622fc4861a4568c3b3f2d75ca`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:27.080670 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8cffb021-5f79-4f08-a9c2-026ba2cdb07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:27.183037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 953/1896] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:27.235141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949440d-20d2-47c0-a78b-f2f29a1841eb admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:27.236810 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3949440d-20d2-47c0-a78b-f2f29a1841eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:27.344331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949440d-20d2-47c0-a78b-f2f29a1841eb admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:27.349004 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c101a-405f-4135-8aca-e5d8b0b93fb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:27.349833 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c101a-405f-4135-8aca-e5d8b0b93fb0 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:27.350293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c101a-405f-4135-8aca-e5d8b0b93fb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:27.350871 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c101a-405f-4135-8aca-e5d8b0b93fb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:27.375150 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c101a-405f-4135-8aca-e5d8b0b93fb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:27.376674 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c101a-405f-4135-8aca-e5d8b0b93fb0 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:27.394212 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 944/1897] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:28.353646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffb021-5f79-4f08-a9c2-026ba2cdb07c None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:28.381865 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:28.410106 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffb021-5f79-4f08-a9c2-026ba2cdb07c None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:28.414795 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bbb65-44c6-440d-a34f-9a5c5ca91ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:28.415446 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bbb65-44c6-440d-a34f-9a5c5ca91ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:28.415732 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bbb65-44c6-440d-a34f-9a5c5ca91ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:28.415994 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bbb65-44c6-440d-a34f-9a5c5ca91ca7 admin admin] PATH_INFO: `/v3/users/404f7e5625764674afdb9d2828aa419c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:28.417120 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bbb65-44c6-440d-a34f-9a5c5ca91ca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=404f7e5625764674afdb9d2828aa419c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:28.419038 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bbb65-44c6-440d-a34f-9a5c5ca91ca7 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:28.683093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-572bbb65-44c6-440d-a34f-9a5c5ca91ca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '404f7e5625764674afdb9d2828aa419c'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:28.691935 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-572bbb65-44c6-440d-a34f-9a5c5ca91ca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '404f7e5625764674afdb9d2828aa419c'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:28.709476 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-572bbb65-44c6-440d-a34f-9a5c5ca91ca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '404f7e5625764674afdb9d2828aa419c'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:28.711506 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-572bbb65-44c6-440d-a34f-9a5c5ca91ca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '404f7e5625764674afdb9d2828aa419c'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:28.712581 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-572bbb65-44c6-440d-a34f-9a5c5ca91ca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '404f7e5625764674afdb9d2828aa419c'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:28.846530 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 954/1898] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:28 2022] DELETE /identity/v3/users/404f7e5625764674afdb9d2828aa419c => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:28.850861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fe6d1-6d72-43e7-9adf-69f182899786 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:28.851439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fe6d1-6d72-43e7-9adf-69f182899786 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:28.851439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fe6d1-6d72-43e7-9adf-69f182899786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:28.873176 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-9d9fe6d1-6d72-43e7-9adf-69f182899786 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:28.875803 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9d9fe6d1-6d72-43e7-9adf-69f182899786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:28.940488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 945/1899] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:28.950490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bbb65-44c6-440d-a34f-9a5c5ca91ca7 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:28.952019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-572bbb65-44c6-440d-a34f-9a5c5ca91ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:29.067146 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bbb65-44c6-440d-a34f-9a5c5ca91ca7 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:29.076451 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9657913-3fdb-426d-bbeb-ed4fe0f6db02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:29.077448 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9657913-3fdb-426d-bbeb-ed4fe0f6db02 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:29.078263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9657913-3fdb-426d-bbeb-ed4fe0f6db02 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:29.078814 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9657913-3fdb-426d-bbeb-ed4fe0f6db02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:29.094432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9657913-3fdb-426d-bbeb-ed4fe0f6db02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:29.096369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9657913-3fdb-426d-bbeb-ed4fe0f6db02 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:29.128358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 955/1900] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:57:29.564722 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fe6d1-6d72-43e7-9adf-69f182899786 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:29.567677 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9d9fe6d1-6d72-43e7-9adf-69f182899786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:29.594537 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fe6d1-6d72-43e7-9adf-69f182899786 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:29.617787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fec50-892a-4f61-8b3c-cb741983c7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:29.617787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fec50-892a-4f61-8b3c-cb741983c7d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:29.624206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fec50-892a-4f61-8b3c-cb741983c7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:29.624760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fec50-892a-4f61-8b3c-cb741983c7d3 admin admin] PATH_INFO: `/v3/projects/924a1b6b958f4d86adc0a9ca294f59c0` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:29.626953 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62fec50-892a-4f61-8b3c-cb741983c7d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=924a1b6b958f4d86adc0a9ca294f59c0)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:29.630509 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62fec50-892a-4f61-8b3c-cb741983c7d3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:29.805561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c62fec50-892a-4f61-8b3c-cb741983c7d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '924a1b6b958f4d86adc0a9ca294f59c0', 'request_id': 'req-c62fec50-892a-4f61-8b3c-cb741983c7d3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:29.812827 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c62fec50-892a-4f61-8b3c-cb741983c7d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '924a1b6b958f4d86adc0a9ca294f59c0', 'request_id': 'req-c62fec50-892a-4f61-8b3c-cb741983c7d3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:29.814632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:29.835060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c62fec50-892a-4f61-8b3c-cb741983c7d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '924a1b6b958f4d86adc0a9ca294f59c0', 'request_id': 'req-c62fec50-892a-4f61-8b3c-cb741983c7d3'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:29.962775 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c62fec50-892a-4f61-8b3c-cb741983c7d3 admin admin] The token cache is being invalidate because project 924a1b6b958f4d86adc0a9ca294f59c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:29.963465 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c62fec50-892a-4f61-8b3c-cb741983c7d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:29.963744 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c62fec50-892a-4f61-8b3c-cb741983c7d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:29.965277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 946/1901] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:29 2022] DELETE /identity/v3/projects/924a1b6b958f4d86adc0a9ca294f59c0 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:29.969493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9657913-3fdb-426d-bbeb-ed4fe0f6db02 service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:29.980815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9657913-3fdb-426d-bbeb-ed4fe0f6db02 service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:30.081257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:30.091301 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd978a4-b9b0-437c-9e9e-fb388516cbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:30.094343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd978a4-b9b0-437c-9e9e-fb388516cbf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:30.094343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd978a4-b9b0-437c-9e9e-fb388516cbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:30.094343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd978a4-b9b0-437c-9e9e-fb388516cbf1 admin admin] PATH_INFO: `/v3/users/a0b6875622fc4861a4568c3b3f2d75ca` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:30.094343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd978a4-b9b0-437c-9e9e-fb388516cbf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0b6875622fc4861a4568c3b3f2d75ca)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:30.095767 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd978a4-b9b0-437c-9e9e-fb388516cbf1 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:30.329348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cfd978a4-b9b0-437c-9e9e-fb388516cbf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0b6875622fc4861a4568c3b3f2d75ca'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:30.372173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cfd978a4-b9b0-437c-9e9e-fb388516cbf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0b6875622fc4861a4568c3b3f2d75ca'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:30.394895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cfd978a4-b9b0-437c-9e9e-fb388516cbf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0b6875622fc4861a4568c3b3f2d75ca'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:30.396266 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cfd978a4-b9b0-437c-9e9e-fb388516cbf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0b6875622fc4861a4568c3b3f2d75ca'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:30.398126 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cfd978a4-b9b0-437c-9e9e-fb388516cbf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0b6875622fc4861a4568c3b3f2d75ca'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:30.473662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 956/1902] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:29 2022] DELETE /identity/v3/users/a0b6875622fc4861a4568c3b3f2d75ca => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:30.941605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fec50-892a-4f61-8b3c-cb741983c7d3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:30.947644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c62fec50-892a-4f61-8b3c-cb741983c7d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:30.980027 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fec50-892a-4f61-8b3c-cb741983c7d3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:31.070870 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67387a42-ce6a-4b9f-a4d1-706a7028d036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:31.071662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67387a42-ce6a-4b9f-a4d1-706a7028d036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:31.072002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67387a42-ce6a-4b9f-a4d1-706a7028d036 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:31.072341 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67387a42-ce6a-4b9f-a4d1-706a7028d036 admin admin] PATH_INFO: `/v3/projects/a9852c5fe67746a2b4e3fc36c015ddb4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:31.073712 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67387a42-ce6a-4b9f-a4d1-706a7028d036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9852c5fe67746a2b4e3fc36c015ddb4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:31.076397 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67387a42-ce6a-4b9f-a4d1-706a7028d036 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:31.210723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-67387a42-ce6a-4b9f-a4d1-706a7028d036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9852c5fe67746a2b4e3fc36c015ddb4', 'request_id': 'req-67387a42-ce6a-4b9f-a4d1-706a7028d036'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:31.220373 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-67387a42-ce6a-4b9f-a4d1-706a7028d036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9852c5fe67746a2b4e3fc36c015ddb4', 'request_id': 'req-67387a42-ce6a-4b9f-a4d1-706a7028d036'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:31.227869 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-67387a42-ce6a-4b9f-a4d1-706a7028d036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9852c5fe67746a2b4e3fc36c015ddb4', 'request_id': 'req-67387a42-ce6a-4b9f-a4d1-706a7028d036'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:31.299732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-67387a42-ce6a-4b9f-a4d1-706a7028d036 admin admin] The token cache is being invalidate because project a9852c5fe67746a2b4e3fc36c015ddb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:31.300116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-67387a42-ce6a-4b9f-a4d1-706a7028d036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:31.300867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-67387a42-ce6a-4b9f-a4d1-706a7028d036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:31.302652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 947/1903] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:30 2022] DELETE /identity/v3/projects/a9852c5fe67746a2b4e3fc36c015ddb4 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:31.307905 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c8b44-f99b-4a48-abba-6a6a889250df None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:31.308431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c8b44-f99b-4a48-abba-6a6a889250df None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:31.308431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c8b44-f99b-4a48-abba-6a6a889250df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:31.330714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-708c8b44-f99b-4a48-abba-6a6a889250df None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:31.331987 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-708c8b44-f99b-4a48-abba-6a6a889250df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:31.401774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 957/1904] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:31.409829 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced43b7-6c2c-4abd-ab2a-b89bdbcf34c5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:31.410187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced43b7-6c2c-4abd-ab2a-b89bdbcf34c5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:31.410532 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced43b7-6c2c-4abd-ab2a-b89bdbcf34c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:31.431517 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6ced43b7-6c2c-4abd-ab2a-b89bdbcf34c5 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:31.432810 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6ced43b7-6c2c-4abd-ab2a-b89bdbcf34c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:31.440887 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 948/1905] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:31.446123 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c8b44-f99b-4a48-abba-6a6a889250df None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:31.456158 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c8b44-f99b-4a48-abba-6a6a889250df None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:31.461514 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c862a-8c33-4a0a-b5d7-68f7b914da22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:31.462116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c862a-8c33-4a0a-b5d7-68f7b914da22 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:31.462393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c862a-8c33-4a0a-b5d7-68f7b914da22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:31.462667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c862a-8c33-4a0a-b5d7-68f7b914da22 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:31.463357 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c862a-8c33-4a0a-b5d7-68f7b914da22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:57:31.463662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c862a-8c33-4a0a-b5d7-68f7b914da22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:31.465288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c862a-8c33-4a0a-b5d7-68f7b914da22 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:31.477245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 958/1906] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:57:31.482192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced43b7-6c2c-4abd-ab2a-b89bdbcf34c5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:31.492007 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced43b7-6c2c-4abd-ab2a-b89bdbcf34c5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:31.497261 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742645e-9f9b-4a92-a762-6532691248e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:31.497865 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742645e-9f9b-4a92-a762-6532691248e8 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:31.498146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742645e-9f9b-4a92-a762-6532691248e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:31.498417 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742645e-9f9b-4a92-a762-6532691248e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:31.499049 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742645e-9f9b-4a92-a762-6532691248e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:31.500967 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742645e-9f9b-4a92-a762-6532691248e8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:31.636399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 949/1907] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:31 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:31.637714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c862a-8c33-4a0a-b5d7-68f7b914da22 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:31.650977 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c862a-8c33-4a0a-b5d7-68f7b914da22 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:31.712230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118b230-eaca-4177-b70a-98084fd75847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:31.712738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118b230-eaca-4177-b70a-98084fd75847 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:31.713018 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118b230-eaca-4177-b70a-98084fd75847 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:31.713291 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118b230-eaca-4177-b70a-98084fd75847 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:31.713956 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118b230-eaca-4177-b70a-98084fd75847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:31.716074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118b230-eaca-4177-b70a-98084fd75847 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:31.724064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:31.842529 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 959/1908] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:31 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:31.846601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742645e-9f9b-4a92-a762-6532691248e8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:31.857015 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742645e-9f9b-4a92-a762-6532691248e8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:31.862072 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3119a56-07aa-44d3-a512-fd62fb324095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:31.862665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3119a56-07aa-44d3-a512-fd62fb324095 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:31.862941 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3119a56-07aa-44d3-a512-fd62fb324095 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:31.863246 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3119a56-07aa-44d3-a512-fd62fb324095 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:31.863887 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3119a56-07aa-44d3-a512-fd62fb324095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:31.865477 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3119a56-07aa-44d3-a512-fd62fb324095 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:31.875606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 950/1909] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:31.879493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118b230-eaca-4177-b70a-98084fd75847 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:31.890119 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118b230-eaca-4177-b70a-98084fd75847 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:31.894744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eccd03-a5e5-42ca-9941-dee5acdd106a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:31.895572 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eccd03-a5e5-42ca-9941-dee5acdd106a admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:31.895848 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eccd03-a5e5-42ca-9941-dee5acdd106a admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:31.896115 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eccd03-a5e5-42ca-9941-dee5acdd106a admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:31.896901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eccd03-a5e5-42ca-9941-dee5acdd106a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:31.898535 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eccd03-a5e5-42ca-9941-dee5acdd106a admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:31.908609 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 960/1910] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:31.912846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3119a56-07aa-44d3-a512-fd62fb324095 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:31.923427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3119a56-07aa-44d3-a512-fd62fb324095 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:31.927914 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56bf8d-e9c3-4432-9bb4-bdb3522cdacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:31.929112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56bf8d-e9c3-4432-9bb4-bdb3522cdacd admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:31.929388 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56bf8d-e9c3-4432-9bb4-bdb3522cdacd admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:31.929658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56bf8d-e9c3-4432-9bb4-bdb3522cdacd admin admin] PATH_INFO: `/v3/projects/04cb170121e8401da0824be35126c3a3/users/80a31a270aef41d1a0cd4792ea3acfdc/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:31.944964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56bf8d-e9c3-4432-9bb4-bdb3522cdacd admin admin] RBAC: Authorizing `identity:create_grant(project_id=04cb170121e8401da0824be35126c3a3, user_id=80a31a270aef41d1a0cd4792ea3acfdc, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:31.947673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56bf8d-e9c3-4432-9bb4-bdb3522cdacd admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:32.023167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 951/1911] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:31 2022] PUT /identity/v3/projects/04cb170121e8401da0824be35126c3a3/users/80a31a270aef41d1a0cd4792ea3acfdc/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:32.035233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723950cd-db55-4fe0-8bb6-adebf29af24b None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:32.035522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723950cd-db55-4fe0-8bb6-adebf29af24b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:32.035796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723950cd-db55-4fe0-8bb6-adebf29af24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:32.064853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-723950cd-db55-4fe0-8bb6-adebf29af24b None None] MFA Rules not processed for user `80a31a270aef41d1a0cd4792ea3acfdc`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:32.065979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-723950cd-db55-4fe0-8bb6-adebf29af24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:32.140548 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 961/1912] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:32.145164 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56bf8d-e9c3-4432-9bb4-bdb3522cdacd admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:32.156303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:32.217131 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56bf8d-e9c3-4432-9bb4-bdb3522cdacd admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:32.222120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a13bb-5221-47da-969a-b465906e53c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:32.222843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a13bb-5221-47da-969a-b465906e53c8 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:32.223062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a13bb-5221-47da-969a-b465906e53c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:32.223368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a13bb-5221-47da-969a-b465906e53c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:32.224456 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a13bb-5221-47da-969a-b465906e53c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:32.226681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a13bb-5221-47da-969a-b465906e53c8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:32.315467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 952/1913] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:32 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:32.319837 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723950cd-db55-4fe0-8bb6-adebf29af24b None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:32.332021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723950cd-db55-4fe0-8bb6-adebf29af24b None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:32.414638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5e1a5-ed6b-43cb-95c9-526eecb36890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:32.415803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5e1a5-ed6b-43cb-95c9-526eecb36890 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:32.416091 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5e1a5-ed6b-43cb-95c9-526eecb36890 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:32.416366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5e1a5-ed6b-43cb-95c9-526eecb36890 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:32.417105 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5e1a5-ed6b-43cb-95c9-526eecb36890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:32.420066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5e1a5-ed6b-43cb-95c9-526eecb36890 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:32.501813 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 962/1914] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:32 2022] POST /identity/v3/users => generated 298 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:32.504478 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a13bb-5221-47da-969a-b465906e53c8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:32.515682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a13bb-5221-47da-969a-b465906e53c8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:32.523670 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd6274-3a2d-4818-9cdd-db5dad046d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:32.524657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd6274-3a2d-4818-9cdd-db5dad046d6e admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:32.525284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd6274-3a2d-4818-9cdd-db5dad046d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:32.525833 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd6274-3a2d-4818-9cdd-db5dad046d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:32.526628 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd6274-3a2d-4818-9cdd-db5dad046d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:32.528466 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd6274-3a2d-4818-9cdd-db5dad046d6e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:32.545424 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 953/1915] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:32.546251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5e1a5-ed6b-43cb-95c9-526eecb36890 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:32.557332 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5e1a5-ed6b-43cb-95c9-526eecb36890 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:32.562596 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17321aba-594e-49f4-929a-0cf83483ddb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:32.563297 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17321aba-594e-49f4-929a-0cf83483ddb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:32.563575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17321aba-594e-49f4-929a-0cf83483ddb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:32.563868 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17321aba-594e-49f4-929a-0cf83483ddb1 admin admin] PATH_INFO: `/v3/projects/2faeedb05a1847c89a5a1c6174e58f66/users/a632a6541c6f4da3809c3f268be97614/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:32.578601 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17321aba-594e-49f4-929a-0cf83483ddb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2faeedb05a1847c89a5a1c6174e58f66, user_id=a632a6541c6f4da3809c3f268be97614, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:32.581305 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17321aba-594e-49f4-929a-0cf83483ddb1 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:32.656910 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 963/1916] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:32 2022] PUT /identity/v3/projects/2faeedb05a1847c89a5a1c6174e58f66/users/a632a6541c6f4da3809c3f268be97614/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:32.660998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd6274-3a2d-4818-9cdd-db5dad046d6e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:32.671798 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd6274-3a2d-4818-9cdd-db5dad046d6e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:32.732992 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28569520-ae5d-4410-b734-f119723d6f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:32.733720 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28569520-ae5d-4410-b734-f119723d6f2c admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:32.733836 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28569520-ae5d-4410-b734-f119723d6f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:32.734040 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28569520-ae5d-4410-b734-f119723d6f2c admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:32.734678 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28569520-ae5d-4410-b734-f119723d6f2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:32.736314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28569520-ae5d-4410-b734-f119723d6f2c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:32.746864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 954/1917] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:32.751268 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17321aba-594e-49f4-929a-0cf83483ddb1 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:32.762698 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17321aba-594e-49f4-929a-0cf83483ddb1 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:32.767338 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256e9e4-b6ef-4157-bc49-fbbc0a1f38ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:32.767981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256e9e4-b6ef-4157-bc49-fbbc0a1f38ba admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:32.768272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256e9e4-b6ef-4157-bc49-fbbc0a1f38ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:32.768541 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256e9e4-b6ef-4157-bc49-fbbc0a1f38ba admin admin] PATH_INFO: `/v3/projects/2faeedb05a1847c89a5a1c6174e58f66/users/a632a6541c6f4da3809c3f268be97614/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:32.770426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256e9e4-b6ef-4157-bc49-fbbc0a1f38ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2faeedb05a1847c89a5a1c6174e58f66, user_id=a632a6541c6f4da3809c3f268be97614, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:32.773110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256e9e4-b6ef-4157-bc49-fbbc0a1f38ba admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:32.783363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 964/1918] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:32 2022] PUT /identity/v3/projects/2faeedb05a1847c89a5a1c6174e58f66/users/a632a6541c6f4da3809c3f268be97614/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:32.793983 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e1e62-a6ed-4445-b0f3-013784f90811 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:32.794275 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e1e62-a6ed-4445-b0f3-013784f90811 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:32.794555 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5e1e62-a6ed-4445-b0f3-013784f90811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:32.825128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ea5e1e62-a6ed-4445-b0f3-013784f90811 None None] MFA Rules not processed for user `a632a6541c6f4da3809c3f268be97614`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:32.826495 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ea5e1e62-a6ed-4445-b0f3-013784f90811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:32.908790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 955/1919] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:32.921491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256e9e4-b6ef-4157-bc49-fbbc0a1f38ba admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:32.924096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9256e9e4-b6ef-4157-bc49-fbbc0a1f38ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:32.956580 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:33.004066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256e9e4-b6ef-4157-bc49-fbbc0a1f38ba admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:33.008519 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a50255-0b93-4f5d-8455-d53cde9b4737 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:33.009205 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a50255-0b93-4f5d-8455-d53cde9b4737 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:33.009486 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a50255-0b93-4f5d-8455-d53cde9b4737 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:33.009903 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a50255-0b93-4f5d-8455-d53cde9b4737 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:33.078383 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a50255-0b93-4f5d-8455-d53cde9b4737 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:33.080261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a50255-0b93-4f5d-8455-d53cde9b4737 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:33.092307 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 965/1920] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:34.216698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e1e62-a6ed-4445-b0f3-013784f90811 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:34.218856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ea5e1e62-a6ed-4445-b0f3-013784f90811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:34.226180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:34.291624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5e1e62-a6ed-4445-b0f3-013784f90811 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:34.296259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76ac83-8abf-4f42-a884-319fcae74ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:34.296898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76ac83-8abf-4f42-a884-319fcae74ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:34.297176 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76ac83-8abf-4f42-a884-319fcae74ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:34.297447 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76ac83-8abf-4f42-a884-319fcae74ed8 admin admin] PATH_INFO: `/v3/users/0a537f6bb4f947b1bfe88bfe7d25787b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:34.298767 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76ac83-8abf-4f42-a884-319fcae74ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a537f6bb4f947b1bfe88bfe7d25787b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:34.300816 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76ac83-8abf-4f42-a884-319fcae74ed8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:34.546653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0c76ac83-8abf-4f42-a884-319fcae74ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a537f6bb4f947b1bfe88bfe7d25787b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:34.556984 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0c76ac83-8abf-4f42-a884-319fcae74ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a537f6bb4f947b1bfe88bfe7d25787b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:34.557771 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0c76ac83-8abf-4f42-a884-319fcae74ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a537f6bb4f947b1bfe88bfe7d25787b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:34.656381 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0c76ac83-8abf-4f42-a884-319fcae74ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a537f6bb4f947b1bfe88bfe7d25787b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:34.672365 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0c76ac83-8abf-4f42-a884-319fcae74ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a537f6bb4f947b1bfe88bfe7d25787b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:34.679435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 956/1921] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:34 2022] DELETE /identity/v3/users/0a537f6bb4f947b1bfe88bfe7d25787b => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:34.685101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1bf16-3e58-4524-ae5b-914e58c0d681 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:34.685575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1bf16-3e58-4524-ae5b-914e58c0d681 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:34.685662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1bf16-3e58-4524-ae5b-914e58c0d681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:34.704667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-c8d1bf16-3e58-4524-ae5b-914e58c0d681 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:34.705931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c8d1bf16-3e58-4524-ae5b-914e58c0d681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:34.777814 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 966/1922] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:34.796737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76ac83-8abf-4f42-a884-319fcae74ed8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:34.798545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0c76ac83-8abf-4f42-a884-319fcae74ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:34.863853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76ac83-8abf-4f42-a884-319fcae74ed8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:34.868593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d0b4f-4cbc-4cf9-8fc1-ea88f14837ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:34.869309 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d0b4f-4cbc-4cf9-8fc1-ea88f14837ea service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:34.869637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d0b4f-4cbc-4cf9-8fc1-ea88f14837ea service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:34.869966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d0b4f-4cbc-4cf9-8fc1-ea88f14837ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:34.889508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d0b4f-4cbc-4cf9-8fc1-ea88f14837ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:34.890926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d0b4f-4cbc-4cf9-8fc1-ea88f14837ea service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:34.900905 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 957/1923] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:35.486923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1bf16-3e58-4524-ae5b-914e58c0d681 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:35.488371 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c8d1bf16-3e58-4524-ae5b-914e58c0d681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:35.503272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1bf16-3e58-4524-ae5b-914e58c0d681 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:35.507697 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e87ad-8785-4311-b5f6-b6db47a7c010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:35.508532 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e87ad-8785-4311-b5f6-b6db47a7c010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:35.508858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e87ad-8785-4311-b5f6-b6db47a7c010 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:35.509206 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e87ad-8785-4311-b5f6-b6db47a7c010 admin admin] PATH_INFO: `/v3/projects/39ac88123be14c39b30b55e87047ad22` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:35.510540 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e87ad-8785-4311-b5f6-b6db47a7c010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ac88123be14c39b30b55e87047ad22)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:35.512862 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e87ad-8785-4311-b5f6-b6db47a7c010 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:35.644164 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-6b3e87ad-8785-4311-b5f6-b6db47a7c010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ac88123be14c39b30b55e87047ad22', 'request_id': 'req-6b3e87ad-8785-4311-b5f6-b6db47a7c010'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:35.650871 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-6b3e87ad-8785-4311-b5f6-b6db47a7c010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ac88123be14c39b30b55e87047ad22', 'request_id': 'req-6b3e87ad-8785-4311-b5f6-b6db47a7c010'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:35.729022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-6b3e87ad-8785-4311-b5f6-b6db47a7c010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ac88123be14c39b30b55e87047ad22', 'request_id': 'req-6b3e87ad-8785-4311-b5f6-b6db47a7c010'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:35.738576 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:35.743571 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-6b3e87ad-8785-4311-b5f6-b6db47a7c010 admin admin] The token cache is being invalidate because project 39ac88123be14c39b30b55e87047ad22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:35.743862 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-6b3e87ad-8785-4311-b5f6-b6db47a7c010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:35.744608 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-6b3e87ad-8785-4311-b5f6-b6db47a7c010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:35.746122 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 967/1924] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:35 2022] DELETE /identity/v3/projects/39ac88123be14c39b30b55e87047ad22 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:35.750042 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d0b4f-4cbc-4cf9-8fc1-ea88f14837ea service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:35.760822 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d0b4f-4cbc-4cf9-8fc1-ea88f14837ea service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:35.819845 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:35.862143 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65f50c-5542-46fd-8e48-87b7ee082e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:35.863699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65f50c-5542-46fd-8e48-87b7ee082e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:35.863699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65f50c-5542-46fd-8e48-87b7ee082e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:35.863699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65f50c-5542-46fd-8e48-87b7ee082e01 admin admin] PATH_INFO: `/v3/users/008b4b5aea0a493192e41bcdeb5e99ba` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:35.865075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65f50c-5542-46fd-8e48-87b7ee082e01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=008b4b5aea0a493192e41bcdeb5e99ba)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:35.867324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65f50c-5542-46fd-8e48-87b7ee082e01 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:36.147808 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9f65f50c-5542-46fd-8e48-87b7ee082e01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '008b4b5aea0a493192e41bcdeb5e99ba'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:36.155504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9f65f50c-5542-46fd-8e48-87b7ee082e01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '008b4b5aea0a493192e41bcdeb5e99ba'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:36.156416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9f65f50c-5542-46fd-8e48-87b7ee082e01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '008b4b5aea0a493192e41bcdeb5e99ba'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:36.233549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9f65f50c-5542-46fd-8e48-87b7ee082e01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '008b4b5aea0a493192e41bcdeb5e99ba'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:36.251022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9f65f50c-5542-46fd-8e48-87b7ee082e01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '008b4b5aea0a493192e41bcdeb5e99ba'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:36.257592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 958/1925] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:35 2022] DELETE /identity/v3/users/008b4b5aea0a493192e41bcdeb5e99ba => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:36.734576 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e87ad-8785-4311-b5f6-b6db47a7c010 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:36.736460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6b3e87ad-8785-4311-b5f6-b6db47a7c010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:36.751901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e87ad-8785-4311-b5f6-b6db47a7c010 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:36.837029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e0e2b-04e4-4404-a1f1-a77a855c13c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:36.838439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e0e2b-04e4-4404-a1f1-a77a855c13c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:36.839043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e0e2b-04e4-4404-a1f1-a77a855c13c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:36.844331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e0e2b-04e4-4404-a1f1-a77a855c13c5 admin admin] PATH_INFO: `/v3/projects/040a3ee0971c47979d420fc505fd6a3d` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:36.844331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e0e2b-04e4-4404-a1f1-a77a855c13c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=040a3ee0971c47979d420fc505fd6a3d)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:36.844331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e0e2b-04e4-4404-a1f1-a77a855c13c5 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:37.022339 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cd4e0e2b-04e4-4404-a1f1-a77a855c13c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '040a3ee0971c47979d420fc505fd6a3d', 'request_id': 'req-cd4e0e2b-04e4-4404-a1f1-a77a855c13c5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:37.029203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cd4e0e2b-04e4-4404-a1f1-a77a855c13c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '040a3ee0971c47979d420fc505fd6a3d', 'request_id': 'req-cd4e0e2b-04e4-4404-a1f1-a77a855c13c5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:37.108032 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cd4e0e2b-04e4-4404-a1f1-a77a855c13c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '040a3ee0971c47979d420fc505fd6a3d', 'request_id': 'req-cd4e0e2b-04e4-4404-a1f1-a77a855c13c5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:37.122561 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cd4e0e2b-04e4-4404-a1f1-a77a855c13c5 admin admin] The token cache is being invalidate because project 040a3ee0971c47979d420fc505fd6a3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:37.122561 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cd4e0e2b-04e4-4404-a1f1-a77a855c13c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:37.123478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-cd4e0e2b-04e4-4404-a1f1-a77a855c13c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:37.125190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 968/1926] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:36 2022] DELETE /identity/v3/projects/040a3ee0971c47979d420fc505fd6a3d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:37.131772 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e88e9-bfc7-4bb3-8d65-a45ce13c80eb None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:37.135758 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e88e9-bfc7-4bb3-8d65-a45ce13c80eb None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:37.135758 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e88e9-bfc7-4bb3-8d65-a45ce13c80eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:37.155226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ab2e88e9-bfc7-4bb3-8d65-a45ce13c80eb None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:37.156436 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ab2e88e9-bfc7-4bb3-8d65-a45ce13c80eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:37.218953 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 959/1927] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:37.229795 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca6e11-9ba5-4cb4-a0e0-7da6d777beaa None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:37.230013 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca6e11-9ba5-4cb4-a0e0-7da6d777beaa None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:37.230288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca6e11-9ba5-4cb4-a0e0-7da6d777beaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:37.282537 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-43ca6e11-9ba5-4cb4-a0e0-7da6d777beaa None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:37.283820 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-43ca6e11-9ba5-4cb4-a0e0-7da6d777beaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:37.293295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 969/1928] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:37.298819 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e88e9-bfc7-4bb3-8d65-a45ce13c80eb None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:37.308515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e88e9-bfc7-4bb3-8d65-a45ce13c80eb None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:37.313139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410acf1e-222d-4c8f-81e7-2f9d17707cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:37.313738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410acf1e-222d-4c8f-81e7-2f9d17707cbf admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:37.314013 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410acf1e-222d-4c8f-81e7-2f9d17707cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:37.314283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410acf1e-222d-4c8f-81e7-2f9d17707cbf admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:37.314926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410acf1e-222d-4c8f-81e7-2f9d17707cbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:57:37.315266 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410acf1e-222d-4c8f-81e7-2f9d17707cbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:37.316856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410acf1e-222d-4c8f-81e7-2f9d17707cbf admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:37.329697 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 960/1929] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:57:37.333397 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca6e11-9ba5-4cb4-a0e0-7da6d777beaa None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:37.343955 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca6e11-9ba5-4cb4-a0e0-7da6d777beaa None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:37.348751 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844e2cd-af55-4b46-a6a3-cbff22eaf0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:37.349344 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844e2cd-af55-4b46-a6a3-cbff22eaf0c3 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:37.349632 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844e2cd-af55-4b46-a6a3-cbff22eaf0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:37.350434 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844e2cd-af55-4b46-a6a3-cbff22eaf0c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:37.351059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844e2cd-af55-4b46-a6a3-cbff22eaf0c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:37.353169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844e2cd-af55-4b46-a6a3-cbff22eaf0c3 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:37.539143 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 970/1930] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:37 2022] POST /identity/v3/projects => generated 371 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:37.540350 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410acf1e-222d-4c8f-81e7-2f9d17707cbf admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:37.550313 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410acf1e-222d-4c8f-81e7-2f9d17707cbf admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:37.583830 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:37.612949 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f08bf-290f-44e8-9585-b6a163e9fe04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:37.613517 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f08bf-290f-44e8-9585-b6a163e9fe04 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:37.613793 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f08bf-290f-44e8-9585-b6a163e9fe04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:37.614058 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f08bf-290f-44e8-9585-b6a163e9fe04 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:37.614722 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f08bf-290f-44e8-9585-b6a163e9fe04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:37.617223 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f08bf-290f-44e8-9585-b6a163e9fe04 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:37.716609 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 961/1931] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:37 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:37.719815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844e2cd-af55-4b46-a6a3-cbff22eaf0c3 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:37.731587 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844e2cd-af55-4b46-a6a3-cbff22eaf0c3 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:37.752130 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4049dc-434a-44ce-b994-daf9c4b8c00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:37.752816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4049dc-434a-44ce-b994-daf9c4b8c00a admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:37.753257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4049dc-434a-44ce-b994-daf9c4b8c00a admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:37.753527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4049dc-434a-44ce-b994-daf9c4b8c00a admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:37.754187 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4049dc-434a-44ce-b994-daf9c4b8c00a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:37.755845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4049dc-434a-44ce-b994-daf9c4b8c00a admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:37.782953 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 971/1932] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:37.787943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f08bf-290f-44e8-9585-b6a163e9fe04 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:37.798718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f08bf-290f-44e8-9585-b6a163e9fe04 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:37.804848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a3547-a794-4a9c-a275-41b11cb7d442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:37.805505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a3547-a794-4a9c-a275-41b11cb7d442 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:37.805827 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a3547-a794-4a9c-a275-41b11cb7d442 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:37.806176 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a3547-a794-4a9c-a275-41b11cb7d442 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:37.806832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a3547-a794-4a9c-a275-41b11cb7d442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:37.808558 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a3547-a794-4a9c-a275-41b11cb7d442 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:37.836633 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 962/1933] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:37.838343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4049dc-434a-44ce-b994-daf9c4b8c00a admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:37.849292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4049dc-434a-44ce-b994-daf9c4b8c00a admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:37.864653 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052d011-96fe-4dfa-894f-55dc53df0d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:37.865369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052d011-96fe-4dfa-894f-55dc53df0d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:37.868662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052d011-96fe-4dfa-894f-55dc53df0d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:37.868662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052d011-96fe-4dfa-894f-55dc53df0d7c admin admin] PATH_INFO: `/v3/projects/ac164c37900d47d8b72f97def8aa4bd2/users/cdf514a51aa44455a54781317192bffd/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:37.881965 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052d011-96fe-4dfa-894f-55dc53df0d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac164c37900d47d8b72f97def8aa4bd2, user_id=cdf514a51aa44455a54781317192bffd, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:37.884754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052d011-96fe-4dfa-894f-55dc53df0d7c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:37.988384 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 972/1934] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:37 2022] PUT /identity/v3/projects/ac164c37900d47d8b72f97def8aa4bd2/users/cdf514a51aa44455a54781317192bffd/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:38.000013 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25a57b-9372-4f7e-81ed-3a98feb36bcb None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:38.000520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25a57b-9372-4f7e-81ed-3a98feb36bcb None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:38.001012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25a57b-9372-4f7e-81ed-3a98feb36bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:38.035047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1e25a57b-9372-4f7e-81ed-3a98feb36bcb None None] MFA Rules not processed for user `cdf514a51aa44455a54781317192bffd`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:38.036228 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1e25a57b-9372-4f7e-81ed-3a98feb36bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:38.098904 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 963/1935] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:38.111040 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052d011-96fe-4dfa-894f-55dc53df0d7c admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:38.112731 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7052d011-96fe-4dfa-894f-55dc53df0d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:38.140758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:38.170517 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052d011-96fe-4dfa-894f-55dc53df0d7c admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:38.174174 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2ced2-d8e1-4c73-ac1c-8bfdb1999991 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:38.174785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2ced2-d8e1-4c73-ac1c-8bfdb1999991 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:38.175062 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2ced2-d8e1-4c73-ac1c-8bfdb1999991 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:38.175373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2ced2-d8e1-4c73-ac1c-8bfdb1999991 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:38.183931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2ced2-d8e1-4c73-ac1c-8bfdb1999991 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:38.185666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2ced2-d8e1-4c73-ac1c-8bfdb1999991 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:38.195423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 973/1936] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:38.221842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25a57b-9372-4f7e-81ed-3a98feb36bcb None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:38.306223 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25a57b-9372-4f7e-81ed-3a98feb36bcb None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:38.312201 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7113837-6f5f-49a3-8f3e-36f7aa94eeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:38.313419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7113837-6f5f-49a3-8f3e-36f7aa94eeee admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:38.313419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7113837-6f5f-49a3-8f3e-36f7aa94eeee admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:38.313419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7113837-6f5f-49a3-8f3e-36f7aa94eeee admin admin] PATH_INFO: `/v3/users/cdf514a51aa44455a54781317192bffd` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:38.314778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7113837-6f5f-49a3-8f3e-36f7aa94eeee admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdf514a51aa44455a54781317192bffd)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:38.316896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7113837-6f5f-49a3-8f3e-36f7aa94eeee admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:38.543346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:38.545221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b7113837-6f5f-49a3-8f3e-36f7aa94eeee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdf514a51aa44455a54781317192bffd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:38.551889 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b7113837-6f5f-49a3-8f3e-36f7aa94eeee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdf514a51aa44455a54781317192bffd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:38.552537 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b7113837-6f5f-49a3-8f3e-36f7aa94eeee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdf514a51aa44455a54781317192bffd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:38.656071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b7113837-6f5f-49a3-8f3e-36f7aa94eeee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdf514a51aa44455a54781317192bffd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:38.671451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b7113837-6f5f-49a3-8f3e-36f7aa94eeee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdf514a51aa44455a54781317192bffd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:38.678488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 964/1937] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:38 2022] DELETE /identity/v3/users/cdf514a51aa44455a54781317192bffd => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:38.682960 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc9963-c04a-47d3-be84-e2bdb17af03c None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:38.683292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc9963-c04a-47d3-be84-e2bdb17af03c None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:38.683547 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc9963-c04a-47d3-be84-e2bdb17af03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:38.703767 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-84fc9963-c04a-47d3-be84-e2bdb17af03c None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:38.709138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-84fc9963-c04a-47d3-be84-e2bdb17af03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:38.774906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 974/1938] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:38.781385 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7113837-6f5f-49a3-8f3e-36f7aa94eeee admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:38.784122 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b7113837-6f5f-49a3-8f3e-36f7aa94eeee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:38.842645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7113837-6f5f-49a3-8f3e-36f7aa94eeee admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:38.846131 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321976e3-57b7-4374-9a34-5d405d615036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:38.846799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321976e3-57b7-4374-9a34-5d405d615036 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:38.847081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321976e3-57b7-4374-9a34-5d405d615036 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:38.847426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321976e3-57b7-4374-9a34-5d405d615036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:38.856246 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321976e3-57b7-4374-9a34-5d405d615036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:38.857934 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321976e3-57b7-4374-9a34-5d405d615036 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:38.868240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 965/1939] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:39.283385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc9963-c04a-47d3-be84-e2bdb17af03c None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:39.285085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-84fc9963-c04a-47d3-be84-e2bdb17af03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:39.298053 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc9963-c04a-47d3-be84-e2bdb17af03c None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:39.302647 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac78b84-84cd-4c32-9d7d-3197578d5d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:39.303844 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac78b84-84cd-4c32-9d7d-3197578d5d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:39.304122 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac78b84-84cd-4c32-9d7d-3197578d5d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:39.304393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac78b84-84cd-4c32-9d7d-3197578d5d1c admin admin] PATH_INFO: `/v3/projects/ac164c37900d47d8b72f97def8aa4bd2` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:39.305512 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac78b84-84cd-4c32-9d7d-3197578d5d1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac164c37900d47d8b72f97def8aa4bd2)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:39.307488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac78b84-84cd-4c32-9d7d-3197578d5d1c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:39.406058 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-fac78b84-84cd-4c32-9d7d-3197578d5d1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac164c37900d47d8b72f97def8aa4bd2', 'request_id': 'req-fac78b84-84cd-4c32-9d7d-3197578d5d1c'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:39.412044 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-fac78b84-84cd-4c32-9d7d-3197578d5d1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac164c37900d47d8b72f97def8aa4bd2', 'request_id': 'req-fac78b84-84cd-4c32-9d7d-3197578d5d1c'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:39.472742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-fac78b84-84cd-4c32-9d7d-3197578d5d1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac164c37900d47d8b72f97def8aa4bd2', 'request_id': 'req-fac78b84-84cd-4c32-9d7d-3197578d5d1c'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:39.485330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-fac78b84-84cd-4c32-9d7d-3197578d5d1c admin admin] The token cache is being invalidate because project ac164c37900d47d8b72f97def8aa4bd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:39.485622 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-fac78b84-84cd-4c32-9d7d-3197578d5d1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:39.486272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-fac78b84-84cd-4c32-9d7d-3197578d5d1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:39.487920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 975/1940] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:39 2022] DELETE /identity/v3/projects/ac164c37900d47d8b72f97def8aa4bd2 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:39.492967 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf3597-d32f-4d6a-b2d7-dc9b70c6b529 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:39.493484 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf3597-d32f-4d6a-b2d7-dc9b70c6b529 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:39.493562 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf3597-d32f-4d6a-b2d7-dc9b70c6b529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:39.514121 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-d6cf3597-d32f-4d6a-b2d7-dc9b70c6b529 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:39.516103 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d6cf3597-d32f-4d6a-b2d7-dc9b70c6b529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:39.579661 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 966/1941] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:39.587822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b00b6c-fcc7-4d29-996f-e9ac74df9544 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:39.588107 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b00b6c-fcc7-4d29-996f-e9ac74df9544 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:39.588379 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b00b6c-fcc7-4d29-996f-e9ac74df9544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:39.615537 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-d0b00b6c-fcc7-4d29-996f-e9ac74df9544 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:39.616661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d0b00b6c-fcc7-4d29-996f-e9ac74df9544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:39.624330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 976/1942] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:39.628498 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf3597-d32f-4d6a-b2d7-dc9b70c6b529 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:39.638100 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf3597-d32f-4d6a-b2d7-dc9b70c6b529 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:39.642578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8b84d-3cf8-4dd7-9ece-66aa7ee2705b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:39.643210 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8b84d-3cf8-4dd7-9ece-66aa7ee2705b admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:39.643487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8b84d-3cf8-4dd7-9ece-66aa7ee2705b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:39.643753 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8b84d-3cf8-4dd7-9ece-66aa7ee2705b admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:39.644424 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8b84d-3cf8-4dd7-9ece-66aa7ee2705b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:57:39.644734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8b84d-3cf8-4dd7-9ece-66aa7ee2705b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:39.646300 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8b84d-3cf8-4dd7-9ece-66aa7ee2705b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:39.657989 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 967/1943] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:57:39.662122 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b00b6c-fcc7-4d29-996f-e9ac74df9544 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:39.671924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b00b6c-fcc7-4d29-996f-e9ac74df9544 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:39.676548 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545b9dc-ed96-4f22-aab9-4b69a9f5d8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:39.677159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545b9dc-ed96-4f22-aab9-4b69a9f5d8bd admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:39.677432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545b9dc-ed96-4f22-aab9-4b69a9f5d8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:39.677731 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545b9dc-ed96-4f22-aab9-4b69a9f5d8bd admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:39.678354 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545b9dc-ed96-4f22-aab9-4b69a9f5d8bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:39.680256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545b9dc-ed96-4f22-aab9-4b69a9f5d8bd admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:39.800675 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 977/1944] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:39 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:39.806142 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8b84d-3cf8-4dd7-9ece-66aa7ee2705b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:39.817561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8b84d-3cf8-4dd7-9ece-66aa7ee2705b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:39.879501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:39.891675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fe9ab-23f9-4b3e-aae3-14ad1ef9f403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:39.892497 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fe9ab-23f9-4b3e-aae3-14ad1ef9f403 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:39.892914 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fe9ab-23f9-4b3e-aae3-14ad1ef9f403 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:39.893318 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fe9ab-23f9-4b3e-aae3-14ad1ef9f403 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:39.894296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4fe9ab-23f9-4b3e-aae3-14ad1ef9f403 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:39.897737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4fe9ab-23f9-4b3e-aae3-14ad1ef9f403 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:39.995939 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 968/1945] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:39 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:39.997270 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545b9dc-ed96-4f22-aab9-4b69a9f5d8bd admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:40.008430 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545b9dc-ed96-4f22-aab9-4b69a9f5d8bd admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:40.012920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c79fdf-4195-4652-8361-c25d3273dfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:40.013519 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c79fdf-4195-4652-8361-c25d3273dfd2 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:40.013857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c79fdf-4195-4652-8361-c25d3273dfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:40.014102 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c79fdf-4195-4652-8361-c25d3273dfd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:40.014732 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c79fdf-4195-4652-8361-c25d3273dfd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:40.016354 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c79fdf-4195-4652-8361-c25d3273dfd2 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:40.027675 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 978/1946] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:40.031757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fe9ab-23f9-4b3e-aae3-14ad1ef9f403 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:40.042094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fe9ab-23f9-4b3e-aae3-14ad1ef9f403 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:40.046831 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a471a-b1a8-4f75-a63a-c1294f4e3b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:40.047455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a471a-b1a8-4f75-a63a-c1294f4e3b00 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:40.047730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a471a-b1a8-4f75-a63a-c1294f4e3b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:40.047996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a471a-b1a8-4f75-a63a-c1294f4e3b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:40.048633 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a471a-b1a8-4f75-a63a-c1294f4e3b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:40.050320 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a471a-b1a8-4f75-a63a-c1294f4e3b00 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:40.061884 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 969/1947] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:40.066143 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c79fdf-4195-4652-8361-c25d3273dfd2 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:40.078267 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c79fdf-4195-4652-8361-c25d3273dfd2 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:40.085099 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be0a49-3189-4155-8db5-65e03eb24b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:40.086249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be0a49-3189-4155-8db5-65e03eb24b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:40.086249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be0a49-3189-4155-8db5-65e03eb24b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:40.086441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be0a49-3189-4155-8db5-65e03eb24b44 admin admin] PATH_INFO: `/v3/projects/8578eba371f848bfbdec59c21dea3ac3/users/24d86dc2e053418895376e6818da582c/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:40.096258 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:40.105975 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be0a49-3189-4155-8db5-65e03eb24b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8578eba371f848bfbdec59c21dea3ac3, user_id=24d86dc2e053418895376e6818da582c, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:40.109065 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be0a49-3189-4155-8db5-65e03eb24b44 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:40.226944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 979/1948] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:40 2022] PUT /identity/v3/projects/8578eba371f848bfbdec59c21dea3ac3/users/24d86dc2e053418895376e6818da582c/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:40.233451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a035ba0-f843-4505-85bd-21cc9abc556a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:40.233753 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a035ba0-f843-4505-85bd-21cc9abc556a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:40.234026 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a035ba0-f843-4505-85bd-21cc9abc556a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:40.272124 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1a035ba0-f843-4505-85bd-21cc9abc556a None None] MFA Rules not processed for user `24d86dc2e053418895376e6818da582c`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:40.273410 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1a035ba0-f843-4505-85bd-21cc9abc556a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:40.347677 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 970/1949] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:40.359148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be0a49-3189-4155-8db5-65e03eb24b44 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:40.360869 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-98be0a49-3189-4155-8db5-65e03eb24b44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:40.440745 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be0a49-3189-4155-8db5-65e03eb24b44 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:40.445870 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4f131-78d2-4b2f-8c8e-ed389d203d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:40.446482 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4f131-78d2-4b2f-8c8e-ed389d203d63 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:40.446980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4f131-78d2-4b2f-8c8e-ed389d203d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:40.447253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4f131-78d2-4b2f-8c8e-ed389d203d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:40.457677 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4f131-78d2-4b2f-8c8e-ed389d203d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:40.459643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4f131-78d2-4b2f-8c8e-ed389d203d63 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:40.469099 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 980/1950] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:40.979684 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a035ba0-f843-4505-85bd-21cc9abc556a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:40.981138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1a035ba0-f843-4505-85bd-21cc9abc556a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:40.994332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a035ba0-f843-4505-85bd-21cc9abc556a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:40.998342 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6513c-9d34-4743-9067-c822d0708b90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:40.999010 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6513c-9d34-4743-9067-c822d0708b90 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:40.999322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6513c-9d34-4743-9067-c822d0708b90 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:40.999594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6513c-9d34-4743-9067-c822d0708b90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:41.001318 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-43e6513c-9d34-4743-9067-c822d0708b90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:41.075271 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6513c-9d34-4743-9067-c822d0708b90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:41.076726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6513c-9d34-4743-9067-c822d0708b90 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:41.087184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 971/1951] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:40 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:45.147795 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4f131-78d2-4b2f-8c8e-ed389d203d63 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:45.149214 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-54f4f131-78d2-4b2f-8c8e-ed389d203d63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:45.216473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4f131-78d2-4b2f-8c8e-ed389d203d63 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:45.221433 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155826a-f941-49e9-b743-0b45b65eb9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:45.224984 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155826a-f941-49e9-b743-0b45b65eb9e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:45.224984 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155826a-f941-49e9-b743-0b45b65eb9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:45.224984 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155826a-f941-49e9-b743-0b45b65eb9e4 admin admin] PATH_INFO: `/v3/users/06fe98653eac47cdb31cdf0928b5ad5a` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:45.224984 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155826a-f941-49e9-b743-0b45b65eb9e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06fe98653eac47cdb31cdf0928b5ad5a)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:45.226334 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155826a-f941-49e9-b743-0b45b65eb9e4 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:45.429093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c155826a-f941-49e9-b743-0b45b65eb9e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06fe98653eac47cdb31cdf0928b5ad5a'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:45.438332 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c155826a-f941-49e9-b743-0b45b65eb9e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06fe98653eac47cdb31cdf0928b5ad5a'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:45.450432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c155826a-f941-49e9-b743-0b45b65eb9e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06fe98653eac47cdb31cdf0928b5ad5a'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:45.451155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c155826a-f941-49e9-b743-0b45b65eb9e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06fe98653eac47cdb31cdf0928b5ad5a'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:45.451785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c155826a-f941-49e9-b743-0b45b65eb9e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06fe98653eac47cdb31cdf0928b5ad5a'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:45.540149 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 981/1952] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:45 2022] DELETE /identity/v3/users/06fe98653eac47cdb31cdf0928b5ad5a => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:45.544834 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ef420-205a-4b53-bb72-4dda4cd925a8 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:45.545507 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ef420-205a-4b53-bb72-4dda4cd925a8 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:45.545602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ef420-205a-4b53-bb72-4dda4cd925a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:45.564751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-fb9ef420-205a-4b53-bb72-4dda4cd925a8 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:45.565958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fb9ef420-205a-4b53-bb72-4dda4cd925a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:45.616743 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:45.628707 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 972/1953] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:45.639297 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155826a-f941-49e9-b743-0b45b65eb9e4 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:45.640695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c155826a-f941-49e9-b743-0b45b65eb9e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:45.652466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:45.698727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155826a-f941-49e9-b743-0b45b65eb9e4 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:45.702347 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ff255-97e9-4874-a608-6a70e5ff3803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:45.702891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ff255-97e9-4874-a608-6a70e5ff3803 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:45.703203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ff255-97e9-4874-a608-6a70e5ff3803 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:45.703661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ff255-97e9-4874-a608-6a70e5ff3803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:45.713270 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ff255-97e9-4874-a608-6a70e5ff3803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:45.714696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ff255-97e9-4874-a608-6a70e5ff3803 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:45.724380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 982/1954] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:46.176836 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ef420-205a-4b53-bb72-4dda4cd925a8 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:46.186393 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fb9ef420-205a-4b53-bb72-4dda4cd925a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:46.207667 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ef420-205a-4b53-bb72-4dda4cd925a8 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:46.213083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecad345-fb23-453e-9049-6d3bbdf435e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:46.213779 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecad345-fb23-453e-9049-6d3bbdf435e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:46.214060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecad345-fb23-453e-9049-6d3bbdf435e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:46.214334 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecad345-fb23-453e-9049-6d3bbdf435e8 admin admin] PATH_INFO: `/v3/projects/bc032c1c32824a83a80f852244dfeff4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:46.215515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecad345-fb23-453e-9049-6d3bbdf435e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc032c1c32824a83a80f852244dfeff4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:46.217493 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecad345-fb23-453e-9049-6d3bbdf435e8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:46.357775 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3ecad345-fb23-453e-9049-6d3bbdf435e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc032c1c32824a83a80f852244dfeff4', 'request_id': 'req-3ecad345-fb23-453e-9049-6d3bbdf435e8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:46.364323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3ecad345-fb23-453e-9049-6d3bbdf435e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc032c1c32824a83a80f852244dfeff4', 'request_id': 'req-3ecad345-fb23-453e-9049-6d3bbdf435e8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:46.371040 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3ecad345-fb23-453e-9049-6d3bbdf435e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc032c1c32824a83a80f852244dfeff4', 'request_id': 'req-3ecad345-fb23-453e-9049-6d3bbdf435e8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:46.441987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3ecad345-fb23-453e-9049-6d3bbdf435e8 admin admin] The token cache is being invalidate because project bc032c1c32824a83a80f852244dfeff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:46.442334 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3ecad345-fb23-453e-9049-6d3bbdf435e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:46.442959 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3ecad345-fb23-453e-9049-6d3bbdf435e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:46.445283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 973/1955] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:46 2022] DELETE /identity/v3/projects/bc032c1c32824a83a80f852244dfeff4 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:46.449158 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ff255-97e9-4874-a608-6a70e5ff3803 service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:46.462203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ff255-97e9-4874-a608-6a70e5ff3803 service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:46.537220 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a896dac-d612-4812-8d5c-9dd32446b312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:46.537990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a896dac-d612-4812-8d5c-9dd32446b312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:46.538449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a896dac-d612-4812-8d5c-9dd32446b312 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:46.538797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a896dac-d612-4812-8d5c-9dd32446b312 admin admin] PATH_INFO: `/v3/users/1a7eee434bcc4d3894fbfc496d267ddc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:46.540133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a896dac-d612-4812-8d5c-9dd32446b312 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a7eee434bcc4d3894fbfc496d267ddc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:46.542505 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a896dac-d612-4812-8d5c-9dd32446b312 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:46.740815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9a896dac-d612-4812-8d5c-9dd32446b312 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a7eee434bcc4d3894fbfc496d267ddc'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:46.749221 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9a896dac-d612-4812-8d5c-9dd32446b312 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a7eee434bcc4d3894fbfc496d267ddc'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:46.762794 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9a896dac-d612-4812-8d5c-9dd32446b312 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a7eee434bcc4d3894fbfc496d267ddc'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:46.763613 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9a896dac-d612-4812-8d5c-9dd32446b312 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a7eee434bcc4d3894fbfc496d267ddc'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:46.764236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9a896dac-d612-4812-8d5c-9dd32446b312 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a7eee434bcc4d3894fbfc496d267ddc'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:46.849735 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 983/1956] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:46 2022] DELETE /identity/v3/users/1a7eee434bcc4d3894fbfc496d267ddc => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:47.173056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecad345-fb23-453e-9049-6d3bbdf435e8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:47.174610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3ecad345-fb23-453e-9049-6d3bbdf435e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:47.188176 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecad345-fb23-453e-9049-6d3bbdf435e8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:47.253771 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9cc561-2ad3-4407-8b9f-d608d38a67df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:47.254335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cc561-2ad3-4407-8b9f-d608d38a67df admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:47.254618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cc561-2ad3-4407-8b9f-d608d38a67df admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:47.254893 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cc561-2ad3-4407-8b9f-d608d38a67df admin admin] PATH_INFO: `/v3/users/80a31a270aef41d1a0cd4792ea3acfdc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:47.256134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9cc561-2ad3-4407-8b9f-d608d38a67df admin admin] RBAC: Authorizing `identity:delete_user(user_id=80a31a270aef41d1a0cd4792ea3acfdc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:47.258638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9cc561-2ad3-4407-8b9f-d608d38a67df admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:47.325959 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a896dac-d612-4812-8d5c-9dd32446b312 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:47.327551 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9a896dac-d612-4812-8d5c-9dd32446b312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:47.341833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a896dac-d612-4812-8d5c-9dd32446b312 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:47.346963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b21c7f-1c06-43e5-a1d3-a40bc889574a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:47.347681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b21c7f-1c06-43e5-a1d3-a40bc889574a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:47.347964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b21c7f-1c06-43e5-a1d3-a40bc889574a admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:47.348332 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b21c7f-1c06-43e5-a1d3-a40bc889574a admin admin] PATH_INFO: `/v3/projects/2d0c96c8150947c58293e0a3cc1f5ab5` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:47.349459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b21c7f-1c06-43e5-a1d3-a40bc889574a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d0c96c8150947c58293e0a3cc1f5ab5)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:47.351566 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b21c7f-1c06-43e5-a1d3-a40bc889574a admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:47.466335 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:47.492169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-19b21c7f-1c06-43e5-a1d3-a40bc889574a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d0c96c8150947c58293e0a3cc1f5ab5', 'request_id': 'req-19b21c7f-1c06-43e5-a1d3-a40bc889574a'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:47.499033 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-19b21c7f-1c06-43e5-a1d3-a40bc889574a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d0c96c8150947c58293e0a3cc1f5ab5', 'request_id': 'req-19b21c7f-1c06-43e5-a1d3-a40bc889574a'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:47.500444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-bf9cc561-2ad3-4407-8b9f-d608d38a67df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80a31a270aef41d1a0cd4792ea3acfdc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:47.510466 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-bf9cc561-2ad3-4407-8b9f-d608d38a67df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80a31a270aef41d1a0cd4792ea3acfdc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:47.511430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-bf9cc561-2ad3-4407-8b9f-d608d38a67df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80a31a270aef41d1a0cd4792ea3acfdc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:47.591527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-19b21c7f-1c06-43e5-a1d3-a40bc889574a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d0c96c8150947c58293e0a3cc1f5ab5', 'request_id': 'req-19b21c7f-1c06-43e5-a1d3-a40bc889574a'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:47.609152 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-19b21c7f-1c06-43e5-a1d3-a40bc889574a admin admin] The token cache is being invalidate because project 2d0c96c8150947c58293e0a3cc1f5ab5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:47.609439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-19b21c7f-1c06-43e5-a1d3-a40bc889574a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:47.610343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-19b21c7f-1c06-43e5-a1d3-a40bc889574a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:47.612020 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 984/1957] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:47 2022] DELETE /identity/v3/projects/2d0c96c8150947c58293e0a3cc1f5ab5 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:47.623143 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79aeeba-5b5e-4e99-b693-64b302589b5d None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:47.623143 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79aeeba-5b5e-4e99-b693-64b302589b5d None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:47.623143 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79aeeba-5b5e-4e99-b693-64b302589b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:47.644746 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-c79aeeba-5b5e-4e99-b693-64b302589b5d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:47.646133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c79aeeba-5b5e-4e99-b693-64b302589b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:47.653110 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-bf9cc561-2ad3-4407-8b9f-d608d38a67df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80a31a270aef41d1a0cd4792ea3acfdc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:47.669826 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:47.672597 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-bf9cc561-2ad3-4407-8b9f-d608d38a67df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80a31a270aef41d1a0cd4792ea3acfdc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:47.685573 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 974/1958] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:47 2022] DELETE /identity/v3/users/80a31a270aef41d1a0cd4792ea3acfdc => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:47.690301 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b10dd-42a2-4c45-bf74-7421fb7eb511 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:47.690738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b10dd-42a2-4c45-bf74-7421fb7eb511 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:47.691938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b10dd-42a2-4c45-bf74-7421fb7eb511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:47.729487 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 985/1959] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:57:47.737601 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375decd-15d7-4f90-92c0-96d2c6cdc922 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:47.738201 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375decd-15d7-4f90-92c0-96d2c6cdc922 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:47.738651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375decd-15d7-4f90-92c0-96d2c6cdc922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:47.745525 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f01b10dd-42a2-4c45-bf74-7421fb7eb511 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:47.746941 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f01b10dd-42a2-4c45-bf74-7421fb7eb511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:47.767357 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-a375decd-15d7-4f90-92c0-96d2c6cdc922 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:47.768628 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a375decd-15d7-4f90-92c0-96d2c6cdc922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:47.888288 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 975/1960] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:47.899362 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b10dd-42a2-4c45-bf74-7421fb7eb511 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:47.900784 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f01b10dd-42a2-4c45-bf74-7421fb7eb511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:47.903229 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 986/1961] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:47.906847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375decd-15d7-4f90-92c0-96d2c6cdc922 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:47.934232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375decd-15d7-4f90-92c0-96d2c6cdc922 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:47.940146 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268bf23-d5c0-4628-9735-ba29e8db16fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:47.940825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268bf23-d5c0-4628-9735-ba29e8db16fa admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:47.941170 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268bf23-d5c0-4628-9735-ba29e8db16fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:47.941508 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268bf23-d5c0-4628-9735-ba29e8db16fa admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:47.942242 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268bf23-d5c0-4628-9735-ba29e8db16fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:57:47.942608 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268bf23-d5c0-4628-9735-ba29e8db16fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:47.944838 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268bf23-d5c0-4628-9735-ba29e8db16fa admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:47.961410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 987/1962] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:57:47.967147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268bf23-d5c0-4628-9735-ba29e8db16fa admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:47.980580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b10dd-42a2-4c45-bf74-7421fb7eb511 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:47.985644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c878b9-8fc9-4530-853d-7695cc9bbf15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:47.986643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c878b9-8fc9-4530-853d-7695cc9bbf15 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:47.987146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c878b9-8fc9-4530-853d-7695cc9bbf15 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:47.987621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c878b9-8fc9-4530-853d-7695cc9bbf15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:47.990271 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268bf23-d5c0-4628-9735-ba29e8db16fa admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:47.996460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ce48c-521e-4dd7-8da6-4ec37224bf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:47.997246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ce48c-521e-4dd7-8da6-4ec37224bf31 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:47.997528 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ce48c-521e-4dd7-8da6-4ec37224bf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:47.997798 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ce48c-521e-4dd7-8da6-4ec37224bf31 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:47.998573 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ce48c-521e-4dd7-8da6-4ec37224bf31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:48.000036 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c878b9-8fc9-4530-853d-7695cc9bbf15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:48.000575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ce48c-521e-4dd7-8da6-4ec37224bf31 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:48.001962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c878b9-8fc9-4530-853d-7695cc9bbf15 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:48.014323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 976/1963] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:57:48.086218 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 988/1964] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:47 2022] POST /identity/v3/projects => generated 367 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:48.092229 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c878b9-8fc9-4530-853d-7695cc9bbf15 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:48.102842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c878b9-8fc9-4530-853d-7695cc9bbf15 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:48.176215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4ce5a-44b8-4cbb-80e4-5bc83fd73065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:48.177557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4ce5a-44b8-4cbb-80e4-5bc83fd73065 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.177557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4ce5a-44b8-4cbb-80e4-5bc83fd73065 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.177557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4ce5a-44b8-4cbb-80e4-5bc83fd73065 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.178490 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4ce5a-44b8-4cbb-80e4-5bc83fd73065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:48.181147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4ce5a-44b8-4cbb-80e4-5bc83fd73065 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:48.266203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 977/1965] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:48 2022] POST /identity/v3/users => generated 284 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:48.272918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ce48c-521e-4dd7-8da6-4ec37224bf31 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:48.285275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ce48c-521e-4dd7-8da6-4ec37224bf31 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:48.291255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5081c4d-ed16-4bca-a604-a7b3ad6a1098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:48.292977 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5081c4d-ed16-4bca-a604-a7b3ad6a1098 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.293325 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5081c4d-ed16-4bca-a604-a7b3ad6a1098 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.293584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5081c4d-ed16-4bca-a604-a7b3ad6a1098 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.294245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5081c4d-ed16-4bca-a604-a7b3ad6a1098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:48.295888 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5081c4d-ed16-4bca-a604-a7b3ad6a1098 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:48.307866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 989/1966] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:48.311548 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4ce5a-44b8-4cbb-80e4-5bc83fd73065 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:48.336523 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4ce5a-44b8-4cbb-80e4-5bc83fd73065 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:48.349686 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c6e9f-6389-4ff3-9d0c-023f24739dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:48.350443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c6e9f-6389-4ff3-9d0c-023f24739dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.351620 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c6e9f-6389-4ff3-9d0c-023f24739dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.351966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c6e9f-6389-4ff3-9d0c-023f24739dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.352697 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c6e9f-6389-4ff3-9d0c-023f24739dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:48.354748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c6e9f-6389-4ff3-9d0c-023f24739dc5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:48.367811 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 978/1967] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:48.373889 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5081c4d-ed16-4bca-a604-a7b3ad6a1098 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:48.388885 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5081c4d-ed16-4bca-a604-a7b3ad6a1098 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:48.395646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9cd85-aae0-469e-98cc-44a9fbfcb5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:48.397776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9cd85-aae0-469e-98cc-44a9fbfcb5a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.398128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9cd85-aae0-469e-98cc-44a9fbfcb5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.398469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9cd85-aae0-469e-98cc-44a9fbfcb5a7 admin admin] PATH_INFO: `/v3/projects/4946e6e8d2be4d17978ff9264514bde9/users/30cb617d275142a080bdfb6a1ef4ffe1/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.402297 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:48.421319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed9cd85-aae0-469e-98cc-44a9fbfcb5a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4946e6e8d2be4d17978ff9264514bde9, user_id=30cb617d275142a080bdfb6a1ef4ffe1, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:48.425601 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed9cd85-aae0-469e-98cc-44a9fbfcb5a7 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:48.577463 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 990/1968] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:48 2022] PUT /identity/v3/projects/4946e6e8d2be4d17978ff9264514bde9/users/30cb617d275142a080bdfb6a1ef4ffe1/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:48.586357 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f6980-06bb-4a99-bf56-4f220b012248 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.586869 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f6980-06bb-4a99-bf56-4f220b012248 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.587326 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f6980-06bb-4a99-bf56-4f220b012248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.621230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-8a1f6980-06bb-4a99-bf56-4f220b012248 None None] MFA Rules not processed for user `30cb617d275142a080bdfb6a1ef4ffe1`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:48.622424 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8a1f6980-06bb-4a99-bf56-4f220b012248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:48.628286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:48.697598 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 979/1969] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:48.701878 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9cd85-aae0-469e-98cc-44a9fbfcb5a7 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:48.714476 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f6980-06bb-4a99-bf56-4f220b012248 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:48.716408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8a1f6980-06bb-4a99-bf56-4f220b012248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:48.795765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f6980-06bb-4a99-bf56-4f220b012248 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:48.800685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:48.801325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.801608 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.801965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d admin admin] PATH_INFO: `/v3/projects/04cb170121e8401da0824be35126c3a3` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.803075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=04cb170121e8401da0824be35126c3a3)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:48.805062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:48.816964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9cd85-aae0-469e-98cc-44a9fbfcb5a7 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:48.822222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b3552-232f-4908-813a-a36b8888a7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:48.822818 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b3552-232f-4908-813a-a36b8888a7b4 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:48.823092 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b3552-232f-4908-813a-a36b8888a7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:48.823393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b3552-232f-4908-813a-a36b8888a7b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:48.824033 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b3552-232f-4908-813a-a36b8888a7b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:48.827199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59b3552-232f-4908-813a-a36b8888a7b4 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:48.946694 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 991/1970] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:48 2022] POST /identity/v3/projects => generated 369 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:48.947728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b3552-232f-4908-813a-a36b8888a7b4 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:48.968469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04cb170121e8401da0824be35126c3a3', 'request_id': 'req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:48.975390 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04cb170121e8401da0824be35126c3a3', 'request_id': 'req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:48.981186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04cb170121e8401da0824be35126c3a3', 'request_id': 'req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:49.008699 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b3552-232f-4908-813a-a36b8888a7b4 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:49.016461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9aca9a-6734-42c3-bec3-206dd79a7157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:49.017096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aca9a-6734-42c3-bec3-206dd79a7157 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.017369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aca9a-6734-42c3-bec3-206dd79a7157 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.017636 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aca9a-6734-42c3-bec3-206dd79a7157 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.018268 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9aca9a-6734-42c3-bec3-206dd79a7157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:49.020193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9aca9a-6734-42c3-bec3-206dd79a7157 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:49.076715 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d admin admin] The token cache is being invalidate because project 04cb170121e8401da0824be35126c3a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:49.077073 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:49.077656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:49.079212 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 980/1971] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:48 2022] DELETE /identity/v3/projects/04cb170121e8401da0824be35126c3a3 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:49.083170 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:49.084616 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:49.108545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 992/1972] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:48 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:49.119221 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9aca9a-6734-42c3-bec3-206dd79a7157 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:49.121055 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-2a9aca9a-6734-42c3-bec3-206dd79a7157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:49.140055 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:49.153281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9aca9a-6734-42c3-bec3-206dd79a7157 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:49.158474 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733d463-88b9-42c9-9fff-fe8c33d59298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:49.159117 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733d463-88b9-42c9-9fff-fe8c33d59298 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.159410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733d463-88b9-42c9-9fff-fe8c33d59298 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.159689 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733d463-88b9-42c9-9fff-fe8c33d59298 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.160343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733d463-88b9-42c9-9fff-fe8c33d59298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:49.161977 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733d463-88b9-42c9-9fff-fe8c33d59298 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:49.163849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392f30d-9ba7-4cf0-898f-2b21f3f90c1d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:49.169259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e65d15-b509-48b7-985d-9f2528819219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:49.174579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e65d15-b509-48b7-985d-9f2528819219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.174579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e65d15-b509-48b7-985d-9f2528819219 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.174579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e65d15-b509-48b7-985d-9f2528819219 admin admin] PATH_INFO: `/v3/users/a632a6541c6f4da3809c3f268be97614` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.174579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e65d15-b509-48b7-985d-9f2528819219 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a632a6541c6f4da3809c3f268be97614)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:49.174934 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 993/1973] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:49.175023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e65d15-b509-48b7-985d-9f2528819219 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:49.177849 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733d463-88b9-42c9-9fff-fe8c33d59298 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:49.189082 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733d463-88b9-42c9-9fff-fe8c33d59298 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:49.195546 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f9da2-842d-499d-ba7e-5e32c08c93eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:49.196197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f9da2-842d-499d-ba7e-5e32c08c93eb admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.196474 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f9da2-842d-499d-ba7e-5e32c08c93eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.196792 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f9da2-842d-499d-ba7e-5e32c08c93eb admin admin] PATH_INFO: `/v3/projects/aecc1e389edd4ca39211da4b3d67634a/users/98aac47b487e4c14a618f898e40ad5f9/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.213049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f9da2-842d-499d-ba7e-5e32c08c93eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=aecc1e389edd4ca39211da4b3d67634a, user_id=98aac47b487e4c14a618f898e40ad5f9, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:49.215811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f9da2-842d-499d-ba7e-5e32c08c93eb admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:49.339251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 994/1974] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:49 2022] PUT /identity/v3/projects/aecc1e389edd4ca39211da4b3d67634a/users/98aac47b487e4c14a618f898e40ad5f9/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:49.343381 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f9da2-842d-499d-ba7e-5e32c08c93eb admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:49.370890 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:49.412209 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f9da2-842d-499d-ba7e-5e32c08c93eb admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:49.417735 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cea8b-6994-43a2-89d9-4be26678d7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:49.418367 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cea8b-6994-43a2-89d9-4be26678d7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.418660 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cea8b-6994-43a2-89d9-4be26678d7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.419031 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cea8b-6994-43a2-89d9-4be26678d7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.419620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cea8b-6994-43a2-89d9-4be26678d7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:49.422119 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cea8b-6994-43a2-89d9-4be26678d7d0 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:49.434380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 995/1975] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:57:49.439836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cea8b-6994-43a2-89d9-4be26678d7d0 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:49.450322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cea8b-6994-43a2-89d9-4be26678d7d0 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:49.465151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d34fb-cf98-47f3-b440-6dea695e26fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:49.465151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d34fb-cf98-47f3-b440-6dea695e26fc admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.465151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d34fb-cf98-47f3-b440-6dea695e26fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.465151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d34fb-cf98-47f3-b440-6dea695e26fc admin admin] PATH_INFO: `/v3/projects/aecc1e389edd4ca39211da4b3d67634a/users/98aac47b487e4c14a618f898e40ad5f9/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.465715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d34fb-cf98-47f3-b440-6dea695e26fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=aecc1e389edd4ca39211da4b3d67634a, user_id=98aac47b487e4c14a618f898e40ad5f9, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:49.465715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d34fb-cf98-47f3-b440-6dea695e26fc admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:49.474321 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 996/1976] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:49 2022] PUT /identity/v3/projects/aecc1e389edd4ca39211da4b3d67634a/users/98aac47b487e4c14a618f898e40ad5f9/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:49.486506 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edae1a-3670-4117-ad74-a48ea493ff08 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.486792 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edae1a-3670-4117-ad74-a48ea493ff08 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.487066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edae1a-3670-4117-ad74-a48ea493ff08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.515921 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-76edae1a-3670-4117-ad74-a48ea493ff08 None None] MFA Rules not processed for user `98aac47b487e4c14a618f898e40ad5f9`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:49.517125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-76edae1a-3670-4117-ad74-a48ea493ff08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:49.573184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d4e65d15-b509-48b7-985d-9f2528819219 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a632a6541c6f4da3809c3f268be97614'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:49.587020 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d4e65d15-b509-48b7-985d-9f2528819219 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a632a6541c6f4da3809c3f268be97614'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:49.587020 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d4e65d15-b509-48b7-985d-9f2528819219 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a632a6541c6f4da3809c3f268be97614'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:49.588440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 997/1977] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:57:49.599896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edae1a-3670-4117-ad74-a48ea493ff08 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:49.601420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-76edae1a-3670-4117-ad74-a48ea493ff08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:49.665669 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edae1a-3670-4117-ad74-a48ea493ff08 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:49.669410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52b140-947c-4d6e-b0dc-6a76012d390a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:49.670015 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52b140-947c-4d6e-b0dc-6a76012d390a service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:49.670292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52b140-947c-4d6e-b0dc-6a76012d390a service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:49.670562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52b140-947c-4d6e-b0dc-6a76012d390a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:49.672342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0d52b140-947c-4d6e-b0dc-6a76012d390a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:49.683719 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52b140-947c-4d6e-b0dc-6a76012d390a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:49.686371 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52b140-947c-4d6e-b0dc-6a76012d390a service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:49.697785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 998/1978] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:57:50.167643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d4e65d15-b509-48b7-985d-9f2528819219 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a632a6541c6f4da3809c3f268be97614'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:50.180795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d4e65d15-b509-48b7-985d-9f2528819219 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a632a6541c6f4da3809c3f268be97614'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:50.187033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 981/1979] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:49 2022] DELETE /identity/v3/users/a632a6541c6f4da3809c3f268be97614 => generated 0 bytes in 1104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:50.189613 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52b140-947c-4d6e-b0dc-6a76012d390a service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:50.191220 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0d52b140-947c-4d6e-b0dc-6a76012d390a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:50.202761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52b140-947c-4d6e-b0dc-6a76012d390a service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:50.277650 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf6a04-25f3-408a-9973-c3d79bb68df8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:50.278439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf6a04-25f3-408a-9973-c3d79bb68df8 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:50.279329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf6a04-25f3-408a-9973-c3d79bb68df8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:50.279329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf6a04-25f3-408a-9973-c3d79bb68df8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:50.285822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-81cf6a04-25f3-408a-9973-c3d79bb68df8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:50.355329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf6a04-25f3-408a-9973-c3d79bb68df8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:50.357048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf6a04-25f3-408a-9973-c3d79bb68df8 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:50.370808 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 999/1980] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:50.438104 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:50.675420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e65d15-b509-48b7-985d-9f2528819219 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:50.675420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d4e65d15-b509-48b7-985d-9f2528819219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:50.744395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e65d15-b509-48b7-985d-9f2528819219 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:50.748957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4b3e9-ce44-4ba9-9d23-3ddcd911b7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:50.749580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4b3e9-ce44-4ba9-9d23-3ddcd911b7a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:50.749860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4b3e9-ce44-4ba9-9d23-3ddcd911b7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:50.750135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4b3e9-ce44-4ba9-9d23-3ddcd911b7a1 admin admin] PATH_INFO: `/v3/projects/2faeedb05a1847c89a5a1c6174e58f66` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:50.751341 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4b3e9-ce44-4ba9-9d23-3ddcd911b7a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2faeedb05a1847c89a5a1c6174e58f66)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:50.753265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4b3e9-ce44-4ba9-9d23-3ddcd911b7a1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:50.905724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-07c4b3e9-ce44-4ba9-9d23-3ddcd911b7a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2faeedb05a1847c89a5a1c6174e58f66', 'request_id': 'req-07c4b3e9-ce44-4ba9-9d23-3ddcd911b7a1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:50.912119 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-07c4b3e9-ce44-4ba9-9d23-3ddcd911b7a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2faeedb05a1847c89a5a1c6174e58f66', 'request_id': 'req-07c4b3e9-ce44-4ba9-9d23-3ddcd911b7a1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:50.916447 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:50.918659 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-07c4b3e9-ce44-4ba9-9d23-3ddcd911b7a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2faeedb05a1847c89a5a1c6174e58f66', 'request_id': 'req-07c4b3e9-ce44-4ba9-9d23-3ddcd911b7a1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:51.004127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-07c4b3e9-ce44-4ba9-9d23-3ddcd911b7a1 admin admin] The token cache is being invalidate because project 2faeedb05a1847c89a5a1c6174e58f66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:51.007303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-07c4b3e9-ce44-4ba9-9d23-3ddcd911b7a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:51.007303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-07c4b3e9-ce44-4ba9-9d23-3ddcd911b7a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:51.007303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 982/1981] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:50 2022] DELETE /identity/v3/projects/2faeedb05a1847c89a5a1c6174e58f66 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:51.013066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa9b4c-a906-4bd6-bf65-61068d359a04 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:51.013604 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa9b4c-a906-4bd6-bf65-61068d359a04 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:51.014069 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa9b4c-a906-4bd6-bf65-61068d359a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:51.048160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-22fa9b4c-a906-4bd6-bf65-61068d359a04 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:51.049383 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-22fa9b4c-a906-4bd6-bf65-61068d359a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:51.123148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1000/1982] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:51.130805 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c31856-d58c-4317-a9e2-4c6cb8bd8586 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:51.139746 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c31856-d58c-4317-a9e2-4c6cb8bd8586 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:51.139746 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c31856-d58c-4317-a9e2-4c6cb8bd8586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:51.165044 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-d5c31856-d58c-4317-a9e2-4c6cb8bd8586 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:51.166502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d5c31856-d58c-4317-a9e2-4c6cb8bd8586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:51.175311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 983/1983] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:57:51.180140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa9b4c-a906-4bd6-bf65-61068d359a04 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:51.190209 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa9b4c-a906-4bd6-bf65-61068d359a04 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:51.196810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da73aeb-8651-4f39-b6fb-17eb45b6d717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:51.196810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da73aeb-8651-4f39-b6fb-17eb45b6d717 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:51.196810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da73aeb-8651-4f39-b6fb-17eb45b6d717 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:51.196810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da73aeb-8651-4f39-b6fb-17eb45b6d717 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:51.197847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da73aeb-8651-4f39-b6fb-17eb45b6d717 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:57:51.197847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da73aeb-8651-4f39-b6fb-17eb45b6d717 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:51.198668 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da73aeb-8651-4f39-b6fb-17eb45b6d717 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:51.211707 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1001/1984] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:57:51.216520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c31856-d58c-4317-a9e2-4c6cb8bd8586 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:51.227444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c31856-d58c-4317-a9e2-4c6cb8bd8586 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:51.248722 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549c73e-6b10-47a6-9ecc-01f967b718f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:51.248722 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549c73e-6b10-47a6-9ecc-01f967b718f1 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:51.251649 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549c73e-6b10-47a6-9ecc-01f967b718f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:51.252179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549c73e-6b10-47a6-9ecc-01f967b718f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:51.253123 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549c73e-6b10-47a6-9ecc-01f967b718f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:51.255615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549c73e-6b10-47a6-9ecc-01f967b718f1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:51.342291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 984/1985] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:51 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:51.343464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da73aeb-8651-4f39-b6fb-17eb45b6d717 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:51.354536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da73aeb-8651-4f39-b6fb-17eb45b6d717 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:51.414057 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff5bdf-9313-4dc9-a572-b505ee21b3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:51.414665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff5bdf-9313-4dc9-a572-b505ee21b3bb admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:51.414943 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff5bdf-9313-4dc9-a572-b505ee21b3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:51.415317 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff5bdf-9313-4dc9-a572-b505ee21b3bb admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:51.415947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff5bdf-9313-4dc9-a572-b505ee21b3bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:51.417988 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff5bdf-9313-4dc9-a572-b505ee21b3bb admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:51.566813 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1002/1986] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:51 2022] POST /identity/v3/users => generated 288 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:51.569191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549c73e-6b10-47a6-9ecc-01f967b718f1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:51.579583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549c73e-6b10-47a6-9ecc-01f967b718f1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:51.584167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf08ae-4456-4c24-b02b-d53a9ec46857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:51.585329 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf08ae-4456-4c24-b02b-d53a9ec46857 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:51.585609 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf08ae-4456-4c24-b02b-d53a9ec46857 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:51.585881 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf08ae-4456-4c24-b02b-d53a9ec46857 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:51.586531 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf08ae-4456-4c24-b02b-d53a9ec46857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:51.588189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf08ae-4456-4c24-b02b-d53a9ec46857 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:51.603442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 985/1987] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:51.604602 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff5bdf-9313-4dc9-a572-b505ee21b3bb admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:51.615416 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff5bdf-9313-4dc9-a572-b505ee21b3bb admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:51.620439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7f35e-c68a-4e53-a043-b84d8ac90c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:51.621064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7f35e-c68a-4e53-a043-b84d8ac90c02 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:51.621327 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7f35e-c68a-4e53-a043-b84d8ac90c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:51.621599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7f35e-c68a-4e53-a043-b84d8ac90c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:51.622251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7f35e-c68a-4e53-a043-b84d8ac90c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:51.623879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7f35e-c68a-4e53-a043-b84d8ac90c02 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:51.629712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:51.638219 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1003/1988] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:57:51.643869 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf08ae-4456-4c24-b02b-d53a9ec46857 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:51.659269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf08ae-4456-4c24-b02b-d53a9ec46857 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:51.664759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e1491-ade9-450b-9b95-53e7790a8bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:51.665537 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e1491-ade9-450b-9b95-53e7790a8bec admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:51.665751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e1491-ade9-450b-9b95-53e7790a8bec admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:51.666025 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e1491-ade9-450b-9b95-53e7790a8bec admin admin] PATH_INFO: `/v3/projects/c446fc7d0712436a8de89961520727c3/users/ad3654a0d68a47ceb26b337def749935/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:51.681858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e1491-ade9-450b-9b95-53e7790a8bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=c446fc7d0712436a8de89961520727c3, user_id=ad3654a0d68a47ceb26b337def749935, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:51.683751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e1491-ade9-450b-9b95-53e7790a8bec admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:51.769310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 986/1989] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:51 2022] PUT /identity/v3/projects/c446fc7d0712436a8de89961520727c3/users/ad3654a0d68a47ceb26b337def749935/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:51.779247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed254e-608b-465e-97ca-15c692e8c322 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:51.780502 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed254e-608b-465e-97ca-15c692e8c322 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:51.780502 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed254e-608b-465e-97ca-15c692e8c322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:51.807622 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-44ed254e-608b-465e-97ca-15c692e8c322 None None] MFA Rules not processed for user `ad3654a0d68a47ceb26b337def749935`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:51.808823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-44ed254e-608b-465e-97ca-15c692e8c322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:51.872310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1004/1990] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:51.884506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e1491-ade9-450b-9b95-53e7790a8bec admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:51.888070 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3c2e1491-ade9-450b-9b95-53e7790a8bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:51.943209 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e1491-ade9-450b-9b95-53e7790a8bec admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:51.947440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dffbf5-f01d-41a7-a1a4-7f35732682e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:51.951053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dffbf5-f01d-41a7-a1a4-7f35732682e2 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:51.951053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dffbf5-f01d-41a7-a1a4-7f35732682e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:51.951053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dffbf5-f01d-41a7-a1a4-7f35732682e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:51.956876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dffbf5-f01d-41a7-a1a4-7f35732682e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:51.958301 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dffbf5-f01d-41a7-a1a4-7f35732682e2 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:51.967778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 987/1991] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:52.876136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed254e-608b-465e-97ca-15c692e8c322 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:52.877702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-44ed254e-608b-465e-97ca-15c692e8c322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:52.912122 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dffbf5-f01d-41a7-a1a4-7f35732682e2 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:52.914319 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e2dffbf5-f01d-41a7-a1a4-7f35732682e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:52.984131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed254e-608b-465e-97ca-15c692e8c322 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:52.989268 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd040b-d2d5-48ea-b4f2-2e16d07ef9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:52.991986 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd040b-d2d5-48ea-b4f2-2e16d07ef9e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:52.991986 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd040b-d2d5-48ea-b4f2-2e16d07ef9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:52.991986 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd040b-d2d5-48ea-b4f2-2e16d07ef9e7 admin admin] PATH_INFO: `/v3/users/800a15ff9e154cf3ae89dc7a4282f247` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:52.995493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd040b-d2d5-48ea-b4f2-2e16d07ef9e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=800a15ff9e154cf3ae89dc7a4282f247)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:52.996421 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd040b-d2d5-48ea-b4f2-2e16d07ef9e7 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:53.051145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dffbf5-f01d-41a7-a1a4-7f35732682e2 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:53.057484 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3428109-3f1b-43b7-8d10-c73015ebd669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:53.058127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3428109-3f1b-43b7-8d10-c73015ebd669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:53.058404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3428109-3f1b-43b7-8d10-c73015ebd669 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:53.058675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3428109-3f1b-43b7-8d10-c73015ebd669 admin admin] PATH_INFO: `/v3/users/24d86dc2e053418895376e6818da582c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:53.059819 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3428109-3f1b-43b7-8d10-c73015ebd669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24d86dc2e053418895376e6818da582c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:53.061737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3428109-3f1b-43b7-8d10-c73015ebd669 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:53.187916 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:53.204587 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-09dd040b-d2d5-48ea-b4f2-2e16d07ef9e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '800a15ff9e154cf3ae89dc7a4282f247'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:53.216104 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-09dd040b-d2d5-48ea-b4f2-2e16d07ef9e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '800a15ff9e154cf3ae89dc7a4282f247'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:53.235821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-09dd040b-d2d5-48ea-b4f2-2e16d07ef9e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '800a15ff9e154cf3ae89dc7a4282f247'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:53.235821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-09dd040b-d2d5-48ea-b4f2-2e16d07ef9e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '800a15ff9e154cf3ae89dc7a4282f247'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:53.235821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-09dd040b-d2d5-48ea-b4f2-2e16d07ef9e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '800a15ff9e154cf3ae89dc7a4282f247'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:53.281262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b3428109-3f1b-43b7-8d10-c73015ebd669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24d86dc2e053418895376e6818da582c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:53.296472 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1005/1992] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:52 2022] DELETE /identity/v3/users/800a15ff9e154cf3ae89dc7a4282f247 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:53.301605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b3428109-3f1b-43b7-8d10-c73015ebd669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24d86dc2e053418895376e6818da582c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:53.301754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0b329-e478-4063-98e0-982b97b23859 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:53.301924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b3428109-3f1b-43b7-8d10-c73015ebd669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24d86dc2e053418895376e6818da582c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:53.302025 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0b329-e478-4063-98e0-982b97b23859 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:53.302485 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0b329-e478-4063-98e0-982b97b23859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:53.313070 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:53.326313 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-75b0b329-e478-4063-98e0-982b97b23859 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:53.329258 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-75b0b329-e478-4063-98e0-982b97b23859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:53.378892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b3428109-3f1b-43b7-8d10-c73015ebd669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24d86dc2e053418895376e6818da582c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:53.392116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b3428109-3f1b-43b7-8d10-c73015ebd669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24d86dc2e053418895376e6818da582c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:53.393081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1006/1993] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:57:53.401025 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 988/1994] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:52 2022] DELETE /identity/v3/users/24d86dc2e053418895376e6818da582c => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:53.405277 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f75812-dfea-4c54-85ea-0913522da5ec None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:53.405277 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f75812-dfea-4c54-85ea-0913522da5ec None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:53.405277 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f75812-dfea-4c54-85ea-0913522da5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:53.406287 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3428109-3f1b-43b7-8d10-c73015ebd669 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:53.407700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b3428109-3f1b-43b7-8d10-c73015ebd669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:53.424658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-d5f75812-dfea-4c54-85ea-0913522da5ec None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:53.425792 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d5f75812-dfea-4c54-85ea-0913522da5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:53.466383 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3428109-3f1b-43b7-8d10-c73015ebd669 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:53.470284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf7e18-c5fc-4029-be24-86fd12902ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:53.471261 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf7e18-c5fc-4029-be24-86fd12902ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:53.471414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf7e18-c5fc-4029-be24-86fd12902ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:53.471766 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf7e18-c5fc-4029-be24-86fd12902ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:53.496502 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1007/1995] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:53.508620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f75812-dfea-4c54-85ea-0913522da5ec None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:53.516809 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f75812-dfea-4c54-85ea-0913522da5ec None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:53.521254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f445fc0-0c4b-4798-85dc-6c73dd40a5d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:53.521967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f445fc0-0c4b-4798-85dc-6c73dd40a5d5 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:53.523177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f445fc0-0c4b-4798-85dc-6c73dd40a5d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:53.523549 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f445fc0-0c4b-4798-85dc-6c73dd40a5d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:53.534137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f445fc0-0c4b-4798-85dc-6c73dd40a5d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:53.535852 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f445fc0-0c4b-4798-85dc-6c73dd40a5d5 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:53.546952 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1008/1996] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:53.550101 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf7e18-c5fc-4029-be24-86fd12902ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:53.554396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf7e18-c5fc-4029-be24-86fd12902ca7 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:53.564652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 989/1997] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:54.331894 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f445fc0-0c4b-4798-85dc-6c73dd40a5d5 service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:54.331894 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-5f445fc0-0c4b-4798-85dc-6c73dd40a5d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:54.353916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f445fc0-0c4b-4798-85dc-6c73dd40a5d5 service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:54.359796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048c710-a32f-4e05-8346-67120a6e6833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:54.361200 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048c710-a32f-4e05-8346-67120a6e6833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:54.361865 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048c710-a32f-4e05-8346-67120a6e6833 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:54.362305 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048c710-a32f-4e05-8346-67120a6e6833 admin admin] PATH_INFO: `/v3/projects/8578eba371f848bfbdec59c21dea3ac3` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:54.363564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048c710-a32f-4e05-8346-67120a6e6833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8578eba371f848bfbdec59c21dea3ac3)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:54.365513 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048c710-a32f-4e05-8346-67120a6e6833 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:54.390911 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf7e18-c5fc-4029-be24-86fd12902ca7 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:54.392594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3bbf7e18-c5fc-4029-be24-86fd12902ca7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:54.421074 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf7e18-c5fc-4029-be24-86fd12902ca7 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:54.422401 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c05ee-3c57-4ac7-8e51-e1fc71c933eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:54.423053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c05ee-3c57-4ac7-8e51-e1fc71c933eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:54.423366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c05ee-3c57-4ac7-8e51-e1fc71c933eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:54.423824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c05ee-3c57-4ac7-8e51-e1fc71c933eb admin admin] PATH_INFO: `/v3/projects/05b7c5a9fa3f4c41bb4700bf58f26d0f` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:54.425118 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c05ee-3c57-4ac7-8e51-e1fc71c933eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=05b7c5a9fa3f4c41bb4700bf58f26d0f)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:54.427256 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c05ee-3c57-4ac7-8e51-e1fc71c933eb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:54.504114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9048c710-a32f-4e05-8346-67120a6e6833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8578eba371f848bfbdec59c21dea3ac3', 'request_id': 'req-9048c710-a32f-4e05-8346-67120a6e6833'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:54.512568 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9048c710-a32f-4e05-8346-67120a6e6833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8578eba371f848bfbdec59c21dea3ac3', 'request_id': 'req-9048c710-a32f-4e05-8346-67120a6e6833'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:54.595031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b60c05ee-3c57-4ac7-8e51-e1fc71c933eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05b7c5a9fa3f4c41bb4700bf58f26d0f', 'request_id': 'req-b60c05ee-3c57-4ac7-8e51-e1fc71c933eb'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:54.597160 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:54.609795 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9048c710-a32f-4e05-8346-67120a6e6833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8578eba371f848bfbdec59c21dea3ac3', 'request_id': 'req-9048c710-a32f-4e05-8346-67120a6e6833'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:54.627123 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b60c05ee-3c57-4ac7-8e51-e1fc71c933eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05b7c5a9fa3f4c41bb4700bf58f26d0f', 'request_id': 'req-b60c05ee-3c57-4ac7-8e51-e1fc71c933eb'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:54.635687 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9048c710-a32f-4e05-8346-67120a6e6833 admin admin] The token cache is being invalidate because project 8578eba371f848bfbdec59c21dea3ac3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:54.635687 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9048c710-a32f-4e05-8346-67120a6e6833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:54.635687 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9048c710-a32f-4e05-8346-67120a6e6833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:54.648062 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1009/1998] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:54 2022] DELETE /identity/v3/projects/8578eba371f848bfbdec59c21dea3ac3 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:54.661255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b60c05ee-3c57-4ac7-8e51-e1fc71c933eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05b7c5a9fa3f4c41bb4700bf58f26d0f', 'request_id': 'req-b60c05ee-3c57-4ac7-8e51-e1fc71c933eb'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:54.765339 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b60c05ee-3c57-4ac7-8e51-e1fc71c933eb admin admin] The token cache is being invalidate because project 05b7c5a9fa3f4c41bb4700bf58f26d0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:54.765642 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b60c05ee-3c57-4ac7-8e51-e1fc71c933eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:54.766313 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b60c05ee-3c57-4ac7-8e51-e1fc71c933eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:54.770788 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 990/1999] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:54 2022] DELETE /identity/v3/projects/05b7c5a9fa3f4c41bb4700bf58f26d0f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:54.774268 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc440f7-24bf-4ef7-91f6-2ada0102056c None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:54.774562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc440f7-24bf-4ef7-91f6-2ada0102056c None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:54.774842 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc440f7-24bf-4ef7-91f6-2ada0102056c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:54.814969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-bcc440f7-24bf-4ef7-91f6-2ada0102056c None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:54.814969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-bcc440f7-24bf-4ef7-91f6-2ada0102056c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:54.825579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:54.905328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1010/2000] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:54.914604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11705475-6497-4943-860b-f6e2d2ff3f6c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:54.914895 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11705475-6497-4943-860b-f6e2d2ff3f6c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:54.915228 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11705475-6497-4943-860b-f6e2d2ff3f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:54.935280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-11705475-6497-4943-860b-f6e2d2ff3f6c None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:54.936538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-11705475-6497-4943-860b-f6e2d2ff3f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:54.944665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 991/2001] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:54.949322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc440f7-24bf-4ef7-91f6-2ada0102056c None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:54.959446 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc440f7-24bf-4ef7-91f6-2ada0102056c None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:54.972413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cec72f-2548-4897-a2b6-5cfdf084a836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:54.972413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cec72f-2548-4897-a2b6-5cfdf084a836 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:54.972413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cec72f-2548-4897-a2b6-5cfdf084a836 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:54.972413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cec72f-2548-4897-a2b6-5cfdf084a836 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:54.981093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cec72f-2548-4897-a2b6-5cfdf084a836 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:57:54.981671 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cec72f-2548-4897-a2b6-5cfdf084a836 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:54.986336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cec72f-2548-4897-a2b6-5cfdf084a836 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:55.018727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1011/2002] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:57:55.020049 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11705475-6497-4943-860b-f6e2d2ff3f6c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:55.041508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11705475-6497-4943-860b-f6e2d2ff3f6c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:55.047499 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bf409-08d2-4320-b48b-d297f27a0f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:55.048127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bf409-08d2-4320-b48b-d297f27a0f8b admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:55.049181 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bf409-08d2-4320-b48b-d297f27a0f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:55.049975 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8bf409-08d2-4320-b48b-d297f27a0f8b admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:55.051332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8bf409-08d2-4320-b48b-d297f27a0f8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:55.054343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8bf409-08d2-4320-b48b-d297f27a0f8b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:55.186356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 992/2003] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:55 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:55.187673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cec72f-2548-4897-a2b6-5cfdf084a836 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:55.199276 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cec72f-2548-4897-a2b6-5cfdf084a836 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:55.311829 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134815c6-de4f-4d75-9516-1cdc3f295e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:55.312803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134815c6-de4f-4d75-9516-1cdc3f295e5e admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:55.313328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134815c6-de4f-4d75-9516-1cdc3f295e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:55.313749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134815c6-de4f-4d75-9516-1cdc3f295e5e admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:55.314609 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134815c6-de4f-4d75-9516-1cdc3f295e5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:55.316823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134815c6-de4f-4d75-9516-1cdc3f295e5e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:55.409358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1012/2004] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:55 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:55.414112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bf409-08d2-4320-b48b-d297f27a0f8b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:55.425008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8bf409-08d2-4320-b48b-d297f27a0f8b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:55.430212 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1cea1-08c2-47eb-bed8-4adf2c92c162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:55.430820 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1cea1-08c2-47eb-bed8-4adf2c92c162 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:55.431130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1cea1-08c2-47eb-bed8-4adf2c92c162 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:55.431400 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1cea1-08c2-47eb-bed8-4adf2c92c162 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:55.432079 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1cea1-08c2-47eb-bed8-4adf2c92c162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:55.433737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1cea1-08c2-47eb-bed8-4adf2c92c162 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:55.447472 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 993/2005] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:57:55.448743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134815c6-de4f-4d75-9516-1cdc3f295e5e admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:55.459404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134815c6-de4f-4d75-9516-1cdc3f295e5e admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:55.468908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a44b2-1f80-460a-b5df-9a69b87f1436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:55.468908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a44b2-1f80-460a-b5df-9a69b87f1436 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:55.468908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a44b2-1f80-460a-b5df-9a69b87f1436 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:55.468908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a44b2-1f80-460a-b5df-9a69b87f1436 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:55.469195 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a44b2-1f80-460a-b5df-9a69b87f1436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:55.469195 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a44b2-1f80-460a-b5df-9a69b87f1436 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:55.480257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1013/2006] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:55.484684 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1cea1-08c2-47eb-bed8-4adf2c92c162 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:55.494858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1cea1-08c2-47eb-bed8-4adf2c92c162 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:55.500523 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe676911-b181-47a7-9eb4-9f5639180986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:55.501182 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe676911-b181-47a7-9eb4-9f5639180986 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:55.501462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe676911-b181-47a7-9eb4-9f5639180986 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:55.502991 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe676911-b181-47a7-9eb4-9f5639180986 admin admin] PATH_INFO: `/v3/projects/8e939ee179374b6b8b8d73a49f2581ae/users/e37813da0c094209a5398f751a7575e8/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:55.521171 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe676911-b181-47a7-9eb4-9f5639180986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e939ee179374b6b8b8d73a49f2581ae, user_id=e37813da0c094209a5398f751a7575e8, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:55.524807 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe676911-b181-47a7-9eb4-9f5639180986 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:55.624594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 994/2007] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:55 2022] PUT /identity/v3/projects/8e939ee179374b6b8b8d73a49f2581ae/users/e37813da0c094209a5398f751a7575e8/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:55.635573 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac50f-77bf-4853-a64a-f7c50e885c31 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:55.636713 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac50f-77bf-4853-a64a-f7c50e885c31 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:55.636713 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac50f-77bf-4853-a64a-f7c50e885c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:55.697266 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-cbeac50f-77bf-4853-a64a-f7c50e885c31 None None] MFA Rules not processed for user `e37813da0c094209a5398f751a7575e8`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:55.698504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-cbeac50f-77bf-4853-a64a-f7c50e885c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:55.790359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1014/2008] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:55.795489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe676911-b181-47a7-9eb4-9f5639180986 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:55.853025 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeac50f-77bf-4853-a64a-f7c50e885c31 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:55.855000 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-cbeac50f-77bf-4853-a64a-f7c50e885c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:55.880240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:55.884221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe676911-b181-47a7-9eb4-9f5639180986 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:55.889001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3add8e4-92d2-4832-80e9-99331c617498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:55.889653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3add8e4-92d2-4832-80e9-99331c617498 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:55.889932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3add8e4-92d2-4832-80e9-99331c617498 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:55.890199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3add8e4-92d2-4832-80e9-99331c617498 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:55.890838 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3add8e4-92d2-4832-80e9-99331c617498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:55.892770 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3add8e4-92d2-4832-80e9-99331c617498 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:55.932681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeac50f-77bf-4853-a64a-f7c50e885c31 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:55.937356 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204eb118-3914-439e-aba3-f5f573568e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:55.938079 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204eb118-3914-439e-aba3-f5f573568e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:55.938330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204eb118-3914-439e-aba3-f5f573568e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:55.938597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204eb118-3914-439e-aba3-f5f573568e06 admin admin] PATH_INFO: `/v3/users/30cb617d275142a080bdfb6a1ef4ffe1` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:55.939813 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204eb118-3914-439e-aba3-f5f573568e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30cb617d275142a080bdfb6a1ef4ffe1)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:55.941759 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204eb118-3914-439e-aba3-f5f573568e06 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:55.993776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 995/2009] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:55 2022] POST /identity/v3/projects => generated 375 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:57:55.997702 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3add8e4-92d2-4832-80e9-99331c617498 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:56.062638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3add8e4-92d2-4832-80e9-99331c617498 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:56.067651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbce60b-f805-498d-acfd-681dbe5e23b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:56.073996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbce60b-f805-498d-acfd-681dbe5e23b5 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:56.073996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbce60b-f805-498d-acfd-681dbe5e23b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:56.073996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbce60b-f805-498d-acfd-681dbe5e23b5 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:56.073996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbce60b-f805-498d-acfd-681dbe5e23b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:56.073996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbce60b-f805-498d-acfd-681dbe5e23b5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:56.079846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:56.135027 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-204eb118-3914-439e-aba3-f5f573568e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30cb617d275142a080bdfb6a1ef4ffe1'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:56.141853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-204eb118-3914-439e-aba3-f5f573568e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30cb617d275142a080bdfb6a1ef4ffe1'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:56.154955 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-204eb118-3914-439e-aba3-f5f573568e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30cb617d275142a080bdfb6a1ef4ffe1'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:56.155715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-204eb118-3914-439e-aba3-f5f573568e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30cb617d275142a080bdfb6a1ef4ffe1'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:56.156403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-204eb118-3914-439e-aba3-f5f573568e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30cb617d275142a080bdfb6a1ef4ffe1'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:56.185626 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 996/2010] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:55 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:56.188858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbce60b-f805-498d-acfd-681dbe5e23b5 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:56.190458 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9bbce60b-f805-498d-acfd-681dbe5e23b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:56.203535 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbce60b-f805-498d-acfd-681dbe5e23b5 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:56.208671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a1638-01a9-4c48-b91e-665909094ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:56.209292 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a1638-01a9-4c48-b91e-665909094ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:56.210305 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a1638-01a9-4c48-b91e-665909094ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:56.211059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a1638-01a9-4c48-b91e-665909094ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:56.212225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a1638-01a9-4c48-b91e-665909094ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:56.213899 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a1638-01a9-4c48-b91e-665909094ee2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:56.226829 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 997/2011] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:57:56.231929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a1638-01a9-4c48-b91e-665909094ee2 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:56.237860 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1015/2012] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:55 2022] DELETE /identity/v3/users/30cb617d275142a080bdfb6a1ef4ffe1 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:56.243660 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85ec84-9e0c-4166-a31e-5d3d89fd48c8 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:56.244029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85ec84-9e0c-4166-a31e-5d3d89fd48c8 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:56.244228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85ec84-9e0c-4166-a31e-5d3d89fd48c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:56.249627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a1638-01a9-4c48-b91e-665909094ee2 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:56.300851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-6a85ec84-9e0c-4166-a31e-5d3d89fd48c8 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:56.302092 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6a85ec84-9e0c-4166-a31e-5d3d89fd48c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:56.367149 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c2043-19a0-4e04-ac14-d599610204a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:56.367991 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c2043-19a0-4e04-ac14-d599610204a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:56.368402 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c2043-19a0-4e04-ac14-d599610204a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:56.368686 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c2043-19a0-4e04-ac14-d599610204a7 admin admin] PATH_INFO: `/v3/projects/18f367dfc9764367b6e9dfe49082ac0a/users/a90b21de083044f8b8626f403cd9bfb5/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:56.392651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c2043-19a0-4e04-ac14-d599610204a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f367dfc9764367b6e9dfe49082ac0a, user_id=a90b21de083044f8b8626f403cd9bfb5, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:56.394692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1016/2013] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:56.395783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c2043-19a0-4e04-ac14-d599610204a7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:56.410043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85ec84-9e0c-4166-a31e-5d3d89fd48c8 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:56.425658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6a85ec84-9e0c-4166-a31e-5d3d89fd48c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:56.457014 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:56.492784 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85ec84-9e0c-4166-a31e-5d3d89fd48c8 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:56.498606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 998/2014] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:56 2022] PUT /identity/v3/projects/18f367dfc9764367b6e9dfe49082ac0a/users/a90b21de083044f8b8626f403cd9bfb5/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:56.500396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c2043-19a0-4e04-ac14-d599610204a7 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:56.570535 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ac474-3aef-4d21-9e74-696bca3d204d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:56.571082 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ac474-3aef-4d21-9e74-696bca3d204d service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:56.571395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ac474-3aef-4d21-9e74-696bca3d204d service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:56.571677 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ac474-3aef-4d21-9e74-696bca3d204d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:56.596235 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c2043-19a0-4e04-ac14-d599610204a7 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:56.601033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80b63f-6c35-4434-b738-dce2224fb902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:56.601636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80b63f-6c35-4434-b738-dce2224fb902 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:56.601912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80b63f-6c35-4434-b738-dce2224fb902 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:56.602182 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80b63f-6c35-4434-b738-dce2224fb902 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:56.602822 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80b63f-6c35-4434-b738-dce2224fb902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:56.604462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80b63f-6c35-4434-b738-dce2224fb902 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:56.615190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 999/2015] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:56.616535 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80b63f-6c35-4434-b738-dce2224fb902 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:56.618154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4c80b63f-6c35-4434-b738-dce2224fb902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:56.630756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80b63f-6c35-4434-b738-dce2224fb902 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:56.636295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5228f61-ebde-47dd-8522-b771acbe523c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:56.637026 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5228f61-ebde-47dd-8522-b771acbe523c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:56.638932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5228f61-ebde-47dd-8522-b771acbe523c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:56.638932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5228f61-ebde-47dd-8522-b771acbe523c admin admin] PATH_INFO: `/v3/users/ad3654a0d68a47ceb26b337def749935` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:56.638932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5228f61-ebde-47dd-8522-b771acbe523c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad3654a0d68a47ceb26b337def749935)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:56.640673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5228f61-ebde-47dd-8522-b771acbe523c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:56.648037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ac474-3aef-4d21-9e74-696bca3d204d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:56.649552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ac474-3aef-4d21-9e74-696bca3d204d service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:56.660155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1017/2016] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:56.661052 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ac474-3aef-4d21-9e74-696bca3d204d service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:56.671494 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ac474-3aef-4d21-9e74-696bca3d204d service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:56.679212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bc9b2-4748-4e1d-923d-1c872647c0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:56.679212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bc9b2-4748-4e1d-923d-1c872647c0eb admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:56.679212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bc9b2-4748-4e1d-923d-1c872647c0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:56.679212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bc9b2-4748-4e1d-923d-1c872647c0eb admin admin] PATH_INFO: `/v3/projects/18f367dfc9764367b6e9dfe49082ac0a/users/a90b21de083044f8b8626f403cd9bfb5/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:56.681291 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2bc9b2-4748-4e1d-923d-1c872647c0eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f367dfc9764367b6e9dfe49082ac0a, user_id=a90b21de083044f8b8626f403cd9bfb5, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:56.684506 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2bc9b2-4748-4e1d-923d-1c872647c0eb admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:56.697275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1018/2017] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:56 2022] PUT /identity/v3/projects/18f367dfc9764367b6e9dfe49082ac0a/users/a90b21de083044f8b8626f403cd9bfb5/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:56.708802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09b0a4-f347-4e94-b7ad-c64f39a4dd90 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:56.709759 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09b0a4-f347-4e94-b7ad-c64f39a4dd90 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:56.710248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09b0a4-f347-4e94-b7ad-c64f39a4dd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:56.752922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-5b09b0a4-f347-4e94-b7ad-c64f39a4dd90 None None] MFA Rules not processed for user `a90b21de083044f8b8626f403cd9bfb5`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:56.754598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-5b09b0a4-f347-4e94-b7ad-c64f39a4dd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:56.823194 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5228f61-ebde-47dd-8522-b771acbe523c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad3654a0d68a47ceb26b337def749935'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:56.832818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:56.838097 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5228f61-ebde-47dd-8522-b771acbe523c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad3654a0d68a47ceb26b337def749935'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:56.839168 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5228f61-ebde-47dd-8522-b771acbe523c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad3654a0d68a47ceb26b337def749935'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:56.875159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1019/2018] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:56.886882 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09b0a4-f347-4e94-b7ad-c64f39a4dd90 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:56.886882 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-5b09b0a4-f347-4e94-b7ad-c64f39a4dd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:56.905247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5228f61-ebde-47dd-8522-b771acbe523c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad3654a0d68a47ceb26b337def749935'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:56.932339 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:56.950329 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5228f61-ebde-47dd-8522-b771acbe523c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad3654a0d68a47ceb26b337def749935'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:56.965323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1000/2019] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:56 2022] DELETE /identity/v3/users/ad3654a0d68a47ceb26b337def749935 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:56.970420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a296c-40f0-4bae-988c-5ad097cc52d1 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:56.970817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a296c-40f0-4bae-988c-5ad097cc52d1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:56.972361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a296c-40f0-4bae-988c-5ad097cc52d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:57.014346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-be8a296c-40f0-4bae-988c-5ad097cc52d1 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:57.017997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-be8a296c-40f0-4bae-988c-5ad097cc52d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:57.046621 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09b0a4-f347-4e94-b7ad-c64f39a4dd90 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:57.131793 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1001/2020] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:57.142647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a296c-40f0-4bae-988c-5ad097cc52d1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:57.144250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-be8a296c-40f0-4bae-988c-5ad097cc52d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:57.148837 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97f047-0709-4568-b63b-a9ee6951a08b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:57.150286 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97f047-0709-4568-b63b-a9ee6951a08b service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:57.150817 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97f047-0709-4568-b63b-a9ee6951a08b service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:57.151116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97f047-0709-4568-b63b-a9ee6951a08b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:57.154417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-4e97f047-0709-4568-b63b-a9ee6951a08b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:57.248790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a296c-40f0-4bae-988c-5ad097cc52d1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:57.256660 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97f047-0709-4568-b63b-a9ee6951a08b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:57.259389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97f047-0709-4568-b63b-a9ee6951a08b service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:57.260582 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f5657-b53a-4067-8f0b-4ea6ca1a3b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:57.266850 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f5657-b53a-4067-8f0b-4ea6ca1a3b4b service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:57.266850 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f5657-b53a-4067-8f0b-4ea6ca1a3b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:57.266850 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f5657-b53a-4067-8f0b-4ea6ca1a3b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:57.283314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f5657-b53a-4067-8f0b-4ea6ca1a3b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:57.289343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f5657-b53a-4067-8f0b-4ea6ca1a3b4b service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:57.289622 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1020/2021] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:57:57.290717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97f047-0709-4568-b63b-a9ee6951a08b service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:57.292510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-4e97f047-0709-4568-b63b-a9ee6951a08b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:57.302695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1002/2022] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:57:57.312019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97f047-0709-4568-b63b-a9ee6951a08b service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:57.324035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fdcd3-b656-4cfc-a798-3323f6551dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:57.326390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fdcd3-b656-4cfc-a798-3323f6551dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:57.326892 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fdcd3-b656-4cfc-a798-3323f6551dba admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:57.327461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fdcd3-b656-4cfc-a798-3323f6551dba admin admin] PATH_INFO: `/v3/projects/4946e6e8d2be4d17978ff9264514bde9` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:57.329266 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56fdcd3-b656-4cfc-a798-3323f6551dba admin admin] RBAC: Authorizing `identity:delete_project(project_id=4946e6e8d2be4d17978ff9264514bde9)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:57.331726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56fdcd3-b656-4cfc-a798-3323f6551dba admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:57.508027 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f56fdcd3-b656-4cfc-a798-3323f6551dba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4946e6e8d2be4d17978ff9264514bde9', 'request_id': 'req-f56fdcd3-b656-4cfc-a798-3323f6551dba'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:57.514393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f56fdcd3-b656-4cfc-a798-3323f6551dba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4946e6e8d2be4d17978ff9264514bde9', 'request_id': 'req-f56fdcd3-b656-4cfc-a798-3323f6551dba'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:57.613149 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f56fdcd3-b656-4cfc-a798-3323f6551dba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4946e6e8d2be4d17978ff9264514bde9', 'request_id': 'req-f56fdcd3-b656-4cfc-a798-3323f6551dba'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:57.629040 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f56fdcd3-b656-4cfc-a798-3323f6551dba admin admin] The token cache is being invalidate because project 4946e6e8d2be4d17978ff9264514bde9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:57.629369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f56fdcd3-b656-4cfc-a798-3323f6551dba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:57.633021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f56fdcd3-b656-4cfc-a798-3323f6551dba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:57.633371 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1021/2023] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:57 2022] DELETE /identity/v3/projects/4946e6e8d2be4d17978ff9264514bde9 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:57.637430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f5657-b53a-4067-8f0b-4ea6ca1a3b4b service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:57.651972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f5657-b53a-4067-8f0b-4ea6ca1a3b4b service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:57.729736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92df017-085d-4fb2-af91-e8117dec521c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:57.730410 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92df017-085d-4fb2-af91-e8117dec521c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:57.730683 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92df017-085d-4fb2-af91-e8117dec521c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:57.730960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92df017-085d-4fb2-af91-e8117dec521c admin admin] PATH_INFO: `/v3/users/98aac47b487e4c14a618f898e40ad5f9` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:57.734208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92df017-085d-4fb2-af91-e8117dec521c admin admin] RBAC: Authorizing `identity:delete_user(user_id=98aac47b487e4c14a618f898e40ad5f9)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:57.736335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92df017-085d-4fb2-af91-e8117dec521c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:57.753835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:57.803881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15a3c8-4bac-4305-b679-09a2cbfaeed6 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:57.804366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15a3c8-4bac-4305-b679-09a2cbfaeed6 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:57.804715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15a3c8-4bac-4305-b679-09a2cbfaeed6 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:57.806426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1022/2024] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 30 14:57:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:57:57.838890 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f15a3c8-4bac-4305-b679-09a2cbfaeed6 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:57.840741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9f15a3c8-4bac-4305-b679-09a2cbfaeed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:57.852555 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:57.865925 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f15a3c8-4bac-4305-b679-09a2cbfaeed6 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:57.872654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35391eb3-112d-4f6d-b716-675ebdd500cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:57.874543 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35391eb3-112d-4f6d-b716-675ebdd500cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:57.874983 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35391eb3-112d-4f6d-b716-675ebdd500cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:57.875407 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35391eb3-112d-4f6d-b716-675ebdd500cc admin admin] PATH_INFO: `/v3/projects/c446fc7d0712436a8de89961520727c3` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:57.876740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35391eb3-112d-4f6d-b716-675ebdd500cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c446fc7d0712436a8de89961520727c3)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:57.878963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35391eb3-112d-4f6d-b716-675ebdd500cc admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:57.990814 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d92df017-085d-4fb2-af91-e8117dec521c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98aac47b487e4c14a618f898e40ad5f9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:57.998387 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d92df017-085d-4fb2-af91-e8117dec521c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98aac47b487e4c14a618f898e40ad5f9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.000541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d92df017-085d-4fb2-af91-e8117dec521c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98aac47b487e4c14a618f898e40ad5f9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.005583 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-35391eb3-112d-4f6d-b716-675ebdd500cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c446fc7d0712436a8de89961520727c3', 'request_id': 'req-35391eb3-112d-4f6d-b716-675ebdd500cc'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.011524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-35391eb3-112d-4f6d-b716-675ebdd500cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c446fc7d0712436a8de89961520727c3', 'request_id': 'req-35391eb3-112d-4f6d-b716-675ebdd500cc'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.094613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d92df017-085d-4fb2-af91-e8117dec521c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98aac47b487e4c14a618f898e40ad5f9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.113143 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d92df017-085d-4fb2-af91-e8117dec521c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98aac47b487e4c14a618f898e40ad5f9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.119709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1003/2025] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:57 2022] DELETE /identity/v3/users/98aac47b487e4c14a618f898e40ad5f9 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:58.123040 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf227c04-dc90-4948-a7be-b291ee7ed96a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:58.123040 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf227c04-dc90-4948-a7be-b291ee7ed96a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:58.123040 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf227c04-dc90-4948-a7be-b291ee7ed96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:58.142579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bf227c04-dc90-4948-a7be-b291ee7ed96a None None] MFA Rules not processed for user `f4f11655cd51438e9652869f74ffa975`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:58.143832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bf227c04-dc90-4948-a7be-b291ee7ed96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:58.163540 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-35391eb3-112d-4f6d-b716-675ebdd500cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c446fc7d0712436a8de89961520727c3', 'request_id': 'req-35391eb3-112d-4f6d-b716-675ebdd500cc'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.177340 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-35391eb3-112d-4f6d-b716-675ebdd500cc admin admin] The token cache is being invalidate because project c446fc7d0712436a8de89961520727c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:58.177714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-35391eb3-112d-4f6d-b716-675ebdd500cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.178951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-35391eb3-112d-4f6d-b716-675ebdd500cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.180730 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1023/2026] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:57 2022] DELETE /identity/v3/projects/c446fc7d0712436a8de89961520727c3 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:58.181550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35391eb3-112d-4f6d-b716-675ebdd500cc admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:58.186128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-35391eb3-112d-4f6d-b716-675ebdd500cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:58.284658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35391eb3-112d-4f6d-b716-675ebdd500cc admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:58.286557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1004/2027] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 30 14:57:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:58.293655 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf227c04-dc90-4948-a7be-b291ee7ed96a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:58.295642 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bf227c04-dc90-4948-a7be-b291ee7ed96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:58.338421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf227c04-dc90-4948-a7be-b291ee7ed96a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:58.345197 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a1203-a450-4a48-8c44-3f1f9995f65a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4f11655cd51438e9652869f74ffa975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:58.346181 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a1203-a450-4a48-8c44-3f1f9995f65a service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:58.346644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a1203-a450-4a48-8c44-3f1f9995f65a service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:58.347087 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a1203-a450-4a48-8c44-3f1f9995f65a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:58.361809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a1203-a450-4a48-8c44-3f1f9995f65a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:58.361809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a1203-a450-4a48-8c44-3f1f9995f65a service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:58.363739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4f62b-0ac0-42eb-acf6-fbc0901a17a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:58.364658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4f62b-0ac0-42eb-acf6-fbc0901a17a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:58.364903 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4f62b-0ac0-42eb-acf6-fbc0901a17a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:58.365183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4f62b-0ac0-42eb-acf6-fbc0901a17a1 admin admin] PATH_INFO: `/v3/users/07c1825673d7451ba7eb551fc1983228` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:58.366954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4f62b-0ac0-42eb-acf6-fbc0901a17a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07c1825673d7451ba7eb551fc1983228)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:58.369134 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4f62b-0ac0-42eb-acf6-fbc0901a17a1 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:58.377200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1005/2028] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:58.672318 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:58.681289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-56b4f62b-0ac0-42eb-acf6-fbc0901a17a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07c1825673d7451ba7eb551fc1983228'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.689050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-56b4f62b-0ac0-42eb-acf6-fbc0901a17a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07c1825673d7451ba7eb551fc1983228'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.696575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a1203-a450-4a48-8c44-3f1f9995f65a service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:58.699236 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2c2a1203-a450-4a48-8c44-3f1f9995f65a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:58.704958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-56b4f62b-0ac0-42eb-acf6-fbc0901a17a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07c1825673d7451ba7eb551fc1983228'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.705767 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-56b4f62b-0ac0-42eb-acf6-fbc0901a17a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07c1825673d7451ba7eb551fc1983228'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.706815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-56b4f62b-0ac0-42eb-acf6-fbc0901a17a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07c1825673d7451ba7eb551fc1983228'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.714488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a1203-a450-4a48-8c44-3f1f9995f65a service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:58.719981 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd491e2d-49ec-4ff0-8cf1-704a76d69f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:58.720624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd491e2d-49ec-4ff0-8cf1-704a76d69f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:58.720976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd491e2d-49ec-4ff0-8cf1-704a76d69f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:58.721560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd491e2d-49ec-4ff0-8cf1-704a76d69f9e admin admin] PATH_INFO: `/v3/projects/aecc1e389edd4ca39211da4b3d67634a` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:58.722645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd491e2d-49ec-4ff0-8cf1-704a76d69f9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aecc1e389edd4ca39211da4b3d67634a)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:58.728062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd491e2d-49ec-4ff0-8cf1-704a76d69f9e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:58.809375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1024/2029] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:57:58 2022] DELETE /identity/v3/users/07c1825673d7451ba7eb551fc1983228 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:57:58.832101 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cd491e2d-49ec-4ff0-8cf1-704a76d69f9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aecc1e389edd4ca39211da4b3d67634a', 'request_id': 'req-cd491e2d-49ec-4ff0-8cf1-704a76d69f9e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.838579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cd491e2d-49ec-4ff0-8cf1-704a76d69f9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aecc1e389edd4ca39211da4b3d67634a', 'request_id': 'req-cd491e2d-49ec-4ff0-8cf1-704a76d69f9e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.844578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cd491e2d-49ec-4ff0-8cf1-704a76d69f9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aecc1e389edd4ca39211da4b3d67634a', 'request_id': 'req-cd491e2d-49ec-4ff0-8cf1-704a76d69f9e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.942058 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:58.948557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cd491e2d-49ec-4ff0-8cf1-704a76d69f9e admin admin] The token cache is being invalidate because project aecc1e389edd4ca39211da4b3d67634a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:58.948854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cd491e2d-49ec-4ff0-8cf1-704a76d69f9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.951825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cd491e2d-49ec-4ff0-8cf1-704a76d69f9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:58.951825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1006/2030] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:58 2022] DELETE /identity/v3/projects/aecc1e389edd4ca39211da4b3d67634a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:58.957296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90756a51-5bf4-47f6-9de9-eba56534d47b None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:58.957611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90756a51-5bf4-47f6-9de9-eba56534d47b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:58.957891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90756a51-5bf4-47f6-9de9-eba56534d47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:58.982619 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-90756a51-5bf4-47f6-9de9-eba56534d47b None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:58.983871 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-90756a51-5bf4-47f6-9de9-eba56534d47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:59.046887 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1025/2031] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:59.055422 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b080a-1761-4821-85df-42f2298100ee None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.055759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b080a-1761-4821-85df-42f2298100ee None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.061295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b080a-1761-4821-85df-42f2298100ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.077222 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bb6b080a-1761-4821-85df-42f2298100ee None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:59.078441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bb6b080a-1761-4821-85df-42f2298100ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:59.084949 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90756a51-5bf4-47f6-9de9-eba56534d47b None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:59.086381 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-90756a51-5bf4-47f6-9de9-eba56534d47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:59.090487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1007/2032] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:57:59.099055 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b080a-1761-4821-85df-42f2298100ee None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:59.116411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90756a51-5bf4-47f6-9de9-eba56534d47b None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:59.116799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b080a-1761-4821-85df-42f2298100ee None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:59.120910 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889218be-ccd7-41f0-9d2c-574abb2dd12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:59.121800 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889218be-ccd7-41f0-9d2c-574abb2dd12d admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.122131 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889218be-ccd7-41f0-9d2c-574abb2dd12d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.124416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889218be-ccd7-41f0-9d2c-574abb2dd12d admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.124416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889218be-ccd7-41f0-9d2c-574abb2dd12d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:57:59.124416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889218be-ccd7-41f0-9d2c-574abb2dd12d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:59.124705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd48910-2eba-4a6f-b443-4e78a94fc36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:59.125022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd48910-2eba-4a6f-b443-4e78a94fc36d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.125306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd48910-2eba-4a6f-b443-4e78a94fc36d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.125575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd48910-2eba-4a6f-b443-4e78a94fc36d admin admin] PATH_INFO: `/v3/projects/4261589fa8ec4b4bb680a7b3636bbcd1` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.126019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889218be-ccd7-41f0-9d2c-574abb2dd12d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:59.126738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd48910-2eba-4a6f-b443-4e78a94fc36d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4261589fa8ec4b4bb680a7b3636bbcd1)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:59.128703 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd48910-2eba-4a6f-b443-4e78a94fc36d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:59.139241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1008/2033] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:57:59.143476 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889218be-ccd7-41f0-9d2c-574abb2dd12d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:59.153488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889218be-ccd7-41f0-9d2c-574abb2dd12d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:59.158356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda1fa6-22fe-4fad-8239-d0d176ea5fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:59.159180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda1fa6-22fe-4fad-8239-d0d176ea5fbd admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.159464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda1fa6-22fe-4fad-8239-d0d176ea5fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.159736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda1fa6-22fe-4fad-8239-d0d176ea5fbd admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.160372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda1fa6-22fe-4fad-8239-d0d176ea5fbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:59.162297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda1fa6-22fe-4fad-8239-d0d176ea5fbd admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:59.255517 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-dcd48910-2eba-4a6f-b443-4e78a94fc36d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4261589fa8ec4b4bb680a7b3636bbcd1', 'request_id': 'req-dcd48910-2eba-4a6f-b443-4e78a94fc36d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:59.266930 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-dcd48910-2eba-4a6f-b443-4e78a94fc36d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4261589fa8ec4b4bb680a7b3636bbcd1', 'request_id': 'req-dcd48910-2eba-4a6f-b443-4e78a94fc36d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:59.324538 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-dcd48910-2eba-4a6f-b443-4e78a94fc36d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4261589fa8ec4b4bb680a7b3636bbcd1', 'request_id': 'req-dcd48910-2eba-4a6f-b443-4e78a94fc36d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:59.331505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1009/2034] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:59 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:59.335935 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda1fa6-22fe-4fad-8239-d0d176ea5fbd admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:59.342076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-dcd48910-2eba-4a6f-b443-4e78a94fc36d admin admin] The token cache is being invalidate because project 4261589fa8ec4b4bb680a7b3636bbcd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:57:59.342365 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-dcd48910-2eba-4a6f-b443-4e78a94fc36d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:59.343245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-dcd48910-2eba-4a6f-b443-4e78a94fc36d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:57:59.344834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1026/2035] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:57:59 2022] DELETE /identity/v3/projects/4261589fa8ec4b4bb680a7b3636bbcd1 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:59.349851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680b793-cc18-49fe-8221-08c8d755d988 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.351287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680b793-cc18-49fe-8221-08c8d755d988 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.351630 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680b793-cc18-49fe-8221-08c8d755d988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.372385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-b680b793-cc18-49fe-8221-08c8d755d988 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:59.375237 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b680b793-cc18-49fe-8221-08c8d755d988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:59.398056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda1fa6-22fe-4fad-8239-d0d176ea5fbd admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:59.403224 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9802274-a07e-44f9-aef6-3dfca41bf884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:59.403859 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9802274-a07e-44f9-aef6-3dfca41bf884 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.404140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9802274-a07e-44f9-aef6-3dfca41bf884 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.404431 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9802274-a07e-44f9-aef6-3dfca41bf884 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.409536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9802274-a07e-44f9-aef6-3dfca41bf884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:59.409653 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:59.409783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9802274-a07e-44f9-aef6-3dfca41bf884 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:59.436426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1027/2036] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:57:59.444567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556d60c-8f4d-49f8-82c4-df66bd81c0c8 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.444847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556d60c-8f4d-49f8-82c4-df66bd81c0c8 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.449807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556d60c-8f4d-49f8-82c4-df66bd81c0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.464566 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-9556d60c-8f4d-49f8-82c4-df66bd81c0c8 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:59.465691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9556d60c-8f4d-49f8-82c4-df66bd81c0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:59.473727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1028/2037] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:57:59.478475 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556d60c-8f4d-49f8-82c4-df66bd81c0c8 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:59.487552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556d60c-8f4d-49f8-82c4-df66bd81c0c8 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:59.492940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eee7be-ba04-4bbd-9e28-01c767de8780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:59.493516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eee7be-ba04-4bbd-9e28-01c767de8780 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.493810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eee7be-ba04-4bbd-9e28-01c767de8780 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.494062 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eee7be-ba04-4bbd-9e28-01c767de8780 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.494704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eee7be-ba04-4bbd-9e28-01c767de8780 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:57:59.495006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eee7be-ba04-4bbd-9e28-01c767de8780 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:59.496618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eee7be-ba04-4bbd-9e28-01c767de8780 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:59.508087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1029/2038] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:57:59.512179 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eee7be-ba04-4bbd-9e28-01c767de8780 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:59.522562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eee7be-ba04-4bbd-9e28-01c767de8780 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:59.527370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6739f-1be5-4065-a26d-7928c1924cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:59.527959 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6739f-1be5-4065-a26d-7928c1924cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.528250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6739f-1be5-4065-a26d-7928c1924cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.528633 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6739f-1be5-4065-a26d-7928c1924cc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.529264 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6739f-1be5-4065-a26d-7928c1924cc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:59.531178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6739f-1be5-4065-a26d-7928c1924cc5 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:59.542737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1010/2039] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:59 2022] POST /identity/v3/users => generated 291 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:59.548124 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9802274-a07e-44f9-aef6-3dfca41bf884 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:59.550334 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f9802274-a07e-44f9-aef6-3dfca41bf884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:59.562936 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9802274-a07e-44f9-aef6-3dfca41bf884 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:59.567714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb048f1-ef8f-42c7-8b25-47923bc641ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:59.568322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb048f1-ef8f-42c7-8b25-47923bc641ac admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.568604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb048f1-ef8f-42c7-8b25-47923bc641ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.568884 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb048f1-ef8f-42c7-8b25-47923bc641ac admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.569529 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb048f1-ef8f-42c7-8b25-47923bc641ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:59.571143 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb048f1-ef8f-42c7-8b25-47923bc641ac admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:59.580962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1011/2040] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:57:59.585305 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb048f1-ef8f-42c7-8b25-47923bc641ac admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:59.594518 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb048f1-ef8f-42c7-8b25-47923bc641ac admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:59.599656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37a69c-597c-415a-bc15-e3b3987e4443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:59.600772 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37a69c-597c-415a-bc15-e3b3987e4443 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.601086 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37a69c-597c-415a-bc15-e3b3987e4443 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.601355 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac37a69c-597c-415a-bc15-e3b3987e4443 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.602058 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37a69c-597c-415a-bc15-e3b3987e4443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:59.603643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac37a69c-597c-415a-bc15-e3b3987e4443 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:59.615467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1012/2041] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:57:59.618604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37a69c-597c-415a-bc15-e3b3987e4443 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:59.628451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac37a69c-597c-415a-bc15-e3b3987e4443 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:59.633147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d7a43-717c-49d4-b8f7-844281a774c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:59.633773 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d7a43-717c-49d4-b8f7-844281a774c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.634054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d7a43-717c-49d4-b8f7-844281a774c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.634328 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d7a43-717c-49d4-b8f7-844281a774c9 admin admin] PATH_INFO: `/v3/projects/02703ffb2f964879b35bbe443a49e848/users/eb26cd4b255641dbaa7eb4d8ba9ebdd4/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.648653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d7a43-717c-49d4-b8f7-844281a774c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02703ffb2f964879b35bbe443a49e848, user_id=eb26cd4b255641dbaa7eb4d8ba9ebdd4, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:59.651306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d7a43-717c-49d4-b8f7-844281a774c9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:59.668112 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1030/2042] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:59 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:59.672231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6739f-1be5-4065-a26d-7928c1924cc5 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:59.723261 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1013/2043] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:59 2022] PUT /identity/v3/projects/02703ffb2f964879b35bbe443a49e848/users/eb26cd4b255641dbaa7eb4d8ba9ebdd4/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:57:59.733064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6739f-1be5-4065-a26d-7928c1924cc5 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:59.735370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ce26c-fbcf-4418-a125-ad55a2ccfb8b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.735664 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ce26c-fbcf-4418-a125-ad55a2ccfb8b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.735944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ce26c-fbcf-4418-a125-ad55a2ccfb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.763373 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f36ce26c-fbcf-4418-a125-ad55a2ccfb8b None None] MFA Rules not processed for user `eb26cd4b255641dbaa7eb4d8ba9ebdd4`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:57:59.764589 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f36ce26c-fbcf-4418-a125-ad55a2ccfb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:57:59.791516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df76fc2-2719-45cb-8da2-7e9584ce886b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:59.792114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df76fc2-2719-45cb-8da2-7e9584ce886b admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.792384 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df76fc2-2719-45cb-8da2-7e9584ce886b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.792658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df76fc2-2719-45cb-8da2-7e9584ce886b admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.793295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df76fc2-2719-45cb-8da2-7e9584ce886b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:59.795254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df76fc2-2719-45cb-8da2-7e9584ce886b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:59.828800 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1014/2044] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:57:59.833153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ce26c-fbcf-4418-a125-ad55a2ccfb8b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:59.843731 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ce26c-fbcf-4418-a125-ad55a2ccfb8b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:59.848369 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678aca14-ef11-4ca5-b21a-0fd97e4e80bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:59.848972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678aca14-ef11-4ca5-b21a-0fd97e4e80bb admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.849250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678aca14-ef11-4ca5-b21a-0fd97e4e80bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.849520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678aca14-ef11-4ca5-b21a-0fd97e4e80bb admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.850787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678aca14-ef11-4ca5-b21a-0fd97e4e80bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:59.852838 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678aca14-ef11-4ca5-b21a-0fd97e4e80bb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:59.881575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1031/2045] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:59 2022] POST /identity/v3/users => generated 287 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:59.885649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df76fc2-2719-45cb-8da2-7e9584ce886b admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:59.895429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df76fc2-2719-45cb-8da2-7e9584ce886b admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:59.900504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b7282-1101-49fd-a0e8-9772a2ef5d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:59.901087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b7282-1101-49fd-a0e8-9772a2ef5d5c admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.901361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b7282-1101-49fd-a0e8-9772a2ef5d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.901626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b7282-1101-49fd-a0e8-9772a2ef5d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.902249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b7282-1101-49fd-a0e8-9772a2ef5d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:59.904151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b7282-1101-49fd-a0e8-9772a2ef5d5c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:59.912050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:59.914397 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1032/2046] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:57:59.918061 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b7282-1101-49fd-a0e8-9772a2ef5d5c admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:59.927197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b7282-1101-49fd-a0e8-9772a2ef5d5c admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:59.932029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c97dde-ab1e-4fba-999b-d9d5d301ff7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:59.935262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c97dde-ab1e-4fba-999b-d9d5d301ff7c admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.935262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c97dde-ab1e-4fba-999b-d9d5d301ff7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.935262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c97dde-ab1e-4fba-999b-d9d5d301ff7c admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.935262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c97dde-ab1e-4fba-999b-d9d5d301ff7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:59.935580 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c97dde-ab1e-4fba-999b-d9d5d301ff7c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:59.946137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1033/2047] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:57:59.949910 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c97dde-ab1e-4fba-999b-d9d5d301ff7c admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:59.960377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c97dde-ab1e-4fba-999b-d9d5d301ff7c admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:59.963521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1015/2048] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:57:59 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:57:59.965146 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a2b10-75e3-4b74-9d0d-f6753457cda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:59.965911 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a2b10-75e3-4b74-9d0d-f6753457cda5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:59.966494 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a2b10-75e3-4b74-9d0d-f6753457cda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:59.966608 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a2b10-75e3-4b74-9d0d-f6753457cda5 admin admin] PATH_INFO: `/v3/projects/699ef780b54c47e393b5a6dac5b33298/users/f898dc7193a540d496e9645f27e8ea6b/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:59.967319 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678aca14-ef11-4ca5-b21a-0fd97e4e80bb admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:59.981962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a2b10-75e3-4b74-9d0d-f6753457cda5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=699ef780b54c47e393b5a6dac5b33298, user_id=f898dc7193a540d496e9645f27e8ea6b, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:59.984618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a2b10-75e3-4b74-9d0d-f6753457cda5 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:00.029388 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678aca14-ef11-4ca5-b21a-0fd97e4e80bb admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:00.034212 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769e2c8-6162-4698-abd0-b7567f77ba4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:00.034893 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769e2c8-6162-4698-abd0-b7567f77ba4d admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:00.035200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769e2c8-6162-4698-abd0-b7567f77ba4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:00.035471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769e2c8-6162-4698-abd0-b7567f77ba4d admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:00.036145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769e2c8-6162-4698-abd0-b7567f77ba4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:00.038136 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769e2c8-6162-4698-abd0-b7567f77ba4d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:00.090104 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1034/2049] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:57:59 2022] PUT /identity/v3/projects/699ef780b54c47e393b5a6dac5b33298/users/f898dc7193a540d496e9645f27e8ea6b/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:00.101676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2471641-0f1f-4167-b76b-cf8774a0a2d4 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:00.101676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2471641-0f1f-4167-b76b-cf8774a0a2d4 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:00.101676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2471641-0f1f-4167-b76b-cf8774a0a2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:00.129293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e2471641-0f1f-4167-b76b-cf8774a0a2d4 None None] MFA Rules not processed for user `f898dc7193a540d496e9645f27e8ea6b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:00.130510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e2471641-0f1f-4167-b76b-cf8774a0a2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:00.193414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1016/2050] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:57:59 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:00.201246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1035/2051] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:00.201397 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769e2c8-6162-4698-abd0-b7567f77ba4d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:00.208212 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769e2c8-6162-4698-abd0-b7567f77ba4d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:00.208403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2471641-0f1f-4167-b76b-cf8774a0a2d4 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:00.211385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e2471641-0f1f-4167-b76b-cf8774a0a2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:00.294556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2471641-0f1f-4167-b76b-cf8774a0a2d4 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:00.294677 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14fdb1-9f67-42d7-96d8-f32496fde3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:00.295250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14fdb1-9f67-42d7-96d8-f32496fde3bb admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:00.295535 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14fdb1-9f67-42d7-96d8-f32496fde3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:00.295807 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14fdb1-9f67-42d7-96d8-f32496fde3bb admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:00.296460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f14fdb1-9f67-42d7-96d8-f32496fde3bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:00.298071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f14fdb1-9f67-42d7-96d8-f32496fde3bb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:00.301214 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4f301-2aea-48f8-a27f-a7d807c6b407 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:00.301819 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4f301-2aea-48f8-a27f-a7d807c6b407 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:00.302123 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4f301-2aea-48f8-a27f-a7d807c6b407 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:00.302390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4f301-2aea-48f8-a27f-a7d807c6b407 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:00.310013 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1017/2052] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:00.312476 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4f301-2aea-48f8-a27f-a7d807c6b407 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:00.314057 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4f301-2aea-48f8-a27f-a7d807c6b407 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:00.314195 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14fdb1-9f67-42d7-96d8-f32496fde3bb admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:00.323801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1036/2053] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:58:00.324555 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14fdb1-9f67-42d7-96d8-f32496fde3bb admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:00.329818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63134cf4-25e4-486e-9f9f-3080e60f305f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:00.330140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63134cf4-25e4-486e-9f9f-3080e60f305f admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:00.330483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63134cf4-25e4-486e-9f9f-3080e60f305f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:00.330847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63134cf4-25e4-486e-9f9f-3080e60f305f admin admin] PATH_INFO: `/v3/projects/a5ac1a712d7b476d831a4f579481b3dc/users/5376633f469743c9b33076301c387120/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:00.349427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63134cf4-25e4-486e-9f9f-3080e60f305f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5ac1a712d7b476d831a4f579481b3dc, user_id=5376633f469743c9b33076301c387120, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:00.352202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63134cf4-25e4-486e-9f9f-3080e60f305f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:00.418893 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1018/2054] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:00 2022] PUT /identity/v3/projects/a5ac1a712d7b476d831a4f579481b3dc/users/5376633f469743c9b33076301c387120/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:00.430083 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4f301-2aea-48f8-a27f-a7d807c6b407 service nova] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:00.436065 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4f301-2aea-48f8-a27f-a7d807c6b407 service nova] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:00.448725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:00.500859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf9a26-afe1-4b9e-9141-f12146655425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:00.501458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf9a26-afe1-4b9e-9141-f12146655425 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:00.501734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf9a26-afe1-4b9e-9141-f12146655425 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:00.502018 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf9a26-afe1-4b9e-9141-f12146655425 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:00.502631 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf9a26-afe1-4b9e-9141-f12146655425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:00.504466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf9a26-afe1-4b9e-9141-f12146655425 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:00.520441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1037/2055] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:00.523093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63134cf4-25e4-486e-9f9f-3080e60f305f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:00.535813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63134cf4-25e4-486e-9f9f-3080e60f305f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:00.541283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84dda6-a195-4900-8c33-42062d248130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:00.541283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84dda6-a195-4900-8c33-42062d248130 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:00.541283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84dda6-a195-4900-8c33-42062d248130 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:00.541283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84dda6-a195-4900-8c33-42062d248130 admin admin] PATH_INFO: `/v3/projects/a5ac1a712d7b476d831a4f579481b3dc/users/5376633f469743c9b33076301c387120/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:00.542499 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84dda6-a195-4900-8c33-42062d248130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5ac1a712d7b476d831a4f579481b3dc, user_id=5376633f469743c9b33076301c387120, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:00.545174 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84dda6-a195-4900-8c33-42062d248130 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:00.545571 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf9a26-afe1-4b9e-9141-f12146655425 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:00.556348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf9a26-afe1-4b9e-9141-f12146655425 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:00.556689 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1019/2056] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:00 2022] PUT /identity/v3/projects/a5ac1a712d7b476d831a4f579481b3dc/users/5376633f469743c9b33076301c387120/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:00.566751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fba128-68a8-492b-8907-9120265cf111 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:00.567030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fba128-68a8-492b-8907-9120265cf111 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:00.567324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fba128-68a8-492b-8907-9120265cf111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:00.599910 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-33fba128-68a8-492b-8907-9120265cf111 None None] MFA Rules not processed for user `5376633f469743c9b33076301c387120`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:00.601002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-33fba128-68a8-492b-8907-9120265cf111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:00.604992 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:00.616453 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb2035-c1d5-4d1f-9459-ba59ca711365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:00.617087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb2035-c1d5-4d1f-9459-ba59ca711365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:00.617357 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb2035-c1d5-4d1f-9459-ba59ca711365 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:00.617624 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb2035-c1d5-4d1f-9459-ba59ca711365 admin admin] PATH_INFO: `/v3/users/f898dc7193a540d496e9645f27e8ea6b` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:00.618762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb2035-c1d5-4d1f-9459-ba59ca711365 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f898dc7193a540d496e9645f27e8ea6b)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:00.620723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb2035-c1d5-4d1f-9459-ba59ca711365 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:00.636909 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:00.669819 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1020/2057] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:00 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:00.671475 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fba128-68a8-492b-8907-9120265cf111 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:00.672857 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-33fba128-68a8-492b-8907-9120265cf111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:00.688565 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fba128-68a8-492b-8907-9120265cf111 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:00.693104 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9cc85-d2fc-4549-8ec6-fafc1d6a0e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:00.693766 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9cc85-d2fc-4549-8ec6-fafc1d6a0e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:00.694042 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9cc85-d2fc-4549-8ec6-fafc1d6a0e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:00.694315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9cc85-d2fc-4549-8ec6-fafc1d6a0e33 admin admin] PATH_INFO: `/v3/users/e37813da0c094209a5398f751a7575e8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:00.695460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9cc85-d2fc-4549-8ec6-fafc1d6a0e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e37813da0c094209a5398f751a7575e8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:00.697389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9cc85-d2fc-4549-8ec6-fafc1d6a0e33 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:00.862675 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-eecb2035-c1d5-4d1f-9459-ba59ca711365 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f898dc7193a540d496e9645f27e8ea6b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:00.870500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-eecb2035-c1d5-4d1f-9459-ba59ca711365 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f898dc7193a540d496e9645f27e8ea6b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:00.882254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-eecb2035-c1d5-4d1f-9459-ba59ca711365 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f898dc7193a540d496e9645f27e8ea6b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:00.882953 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-eecb2035-c1d5-4d1f-9459-ba59ca711365 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f898dc7193a540d496e9645f27e8ea6b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:00.883609 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-eecb2035-c1d5-4d1f-9459-ba59ca711365 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f898dc7193a540d496e9645f27e8ea6b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:00.931247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5c9cc85-d2fc-4549-8ec6-fafc1d6a0e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e37813da0c094209a5398f751a7575e8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:00.938063 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5c9cc85-d2fc-4549-8ec6-fafc1d6a0e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e37813da0c094209a5398f751a7575e8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:00.938740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5c9cc85-d2fc-4549-8ec6-fafc1d6a0e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e37813da0c094209a5398f751a7575e8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:00.991646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1038/2058] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:00 2022] DELETE /identity/v3/users/f898dc7193a540d496e9645f27e8ea6b => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:00.993331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb2035-c1d5-4d1f-9459-ba59ca711365 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:00.995023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-eecb2035-c1d5-4d1f-9459-ba59ca711365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:01.042966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5c9cc85-d2fc-4549-8ec6-fafc1d6a0e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e37813da0c094209a5398f751a7575e8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:01.052125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb2035-c1d5-4d1f-9459-ba59ca711365 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:01.056275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85b588-48bf-4d91-a883-14eb3793335f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:01.056658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d5c9cc85-d2fc-4549-8ec6-fafc1d6a0e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e37813da0c094209a5398f751a7575e8'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:01.056877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85b588-48bf-4d91-a883-14eb3793335f service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:01.057158 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85b588-48bf-4d91-a883-14eb3793335f service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:01.057431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85b588-48bf-4d91-a883-14eb3793335f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:01.059615 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-4d85b588-48bf-4d91-a883-14eb3793335f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:01.062781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1021/2059] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:00 2022] DELETE /identity/v3/users/e37813da0c094209a5398f751a7575e8 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:01.064160 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373cd1f-d522-4bc3-8fd4-7c01651d86df None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:01.064602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373cd1f-d522-4bc3-8fd4-7c01651d86df None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:01.064988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373cd1f-d522-4bc3-8fd4-7c01651d86df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:01.084893 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-d373cd1f-d522-4bc3-8fd4-7c01651d86df None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:01.086090 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d373cd1f-d522-4bc3-8fd4-7c01651d86df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:01.091428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:01.124838 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85b588-48bf-4d91-a883-14eb3793335f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:01.126187 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85b588-48bf-4d91-a883-14eb3793335f service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:01.137226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1039/2060] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:01.138694 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09785c21-b67b-4ee7-adc2-36ddc9a1c861 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:01.139052 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09785c21-b67b-4ee7-adc2-36ddc9a1c861 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:01.139414 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09785c21-b67b-4ee7-adc2-36ddc9a1c861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:01.149049 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1022/2061] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:01.159286 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-09785c21-b67b-4ee7-adc2-36ddc9a1c861 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:01.160136 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373cd1f-d522-4bc3-8fd4-7c01651d86df None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:01.160493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-09785c21-b67b-4ee7-adc2-36ddc9a1c861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:01.161515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d373cd1f-d522-4bc3-8fd4-7c01651d86df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:01.170343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1040/2062] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:01.180920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09785c21-b67b-4ee7-adc2-36ddc9a1c861 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:01.181661 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:01.182961 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-09785c21-b67b-4ee7-adc2-36ddc9a1c861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:01.241457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373cd1f-d522-4bc3-8fd4-7c01651d86df None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:01.245822 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21355542-6716-4dcf-9b02-74bd61a54d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:01.245958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09785c21-b67b-4ee7-adc2-36ddc9a1c861 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:01.246444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21355542-6716-4dcf-9b02-74bd61a54d55 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:01.246725 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21355542-6716-4dcf-9b02-74bd61a54d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:01.246994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21355542-6716-4dcf-9b02-74bd61a54d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:01.249782 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21a07f-7cc5-4ea6-ba7f-39c178629f83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:01.250382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21a07f-7cc5-4ea6-ba7f-39c178629f83 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:01.250660 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21a07f-7cc5-4ea6-ba7f-39c178629f83 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:01.250927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21a07f-7cc5-4ea6-ba7f-39c178629f83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:01.257180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21355542-6716-4dcf-9b02-74bd61a54d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:01.259309 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21355542-6716-4dcf-9b02-74bd61a54d55 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:01.260134 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21a07f-7cc5-4ea6-ba7f-39c178629f83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:01.261593 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21a07f-7cc5-4ea6-ba7f-39c178629f83 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:01.272620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1041/2063] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:01.273813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1023/2064] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:58:01.834788 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21a07f-7cc5-4ea6-ba7f-39c178629f83 service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:01.836545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ee21a07f-7cc5-4ea6-ba7f-39c178629f83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:01.849215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21a07f-7cc5-4ea6-ba7f-39c178629f83 service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:01.855233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:01.865172 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:01.866283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:01.866283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9 admin admin] PATH_INFO: `/v3/projects/8e939ee179374b6b8b8d73a49f2581ae` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:01.867823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e939ee179374b6b8b8d73a49f2581ae)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:01.869836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:01.970257 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21355542-6716-4dcf-9b02-74bd61a54d55 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:01.973181 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-21355542-6716-4dcf-9b02-74bd61a54d55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:01.987316 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e939ee179374b6b8b8d73a49f2581ae', 'request_id': 'req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:01.988178 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21355542-6716-4dcf-9b02-74bd61a54d55 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:01.994590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e939ee179374b6b8b8d73a49f2581ae', 'request_id': 'req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:02.048604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:02.049238 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:02.049512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:02.049781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93 admin admin] PATH_INFO: `/v3/projects/699ef780b54c47e393b5a6dac5b33298` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:02.050891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=699ef780b54c47e393b5a6dac5b33298)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:02.053438 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:02.087275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e939ee179374b6b8b8d73a49f2581ae', 'request_id': 'req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:02.098999 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9 admin admin] The token cache is being invalidate because project 8e939ee179374b6b8b8d73a49f2581ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:02.099320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:02.099997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:02.101489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1042/2065] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:01 2022] DELETE /identity/v3/projects/8e939ee179374b6b8b8d73a49f2581ae => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:02.105598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:02.107234 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:02.120866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24eaf6-1fe7-45ce-9cda-230d106e9bb9 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:02.126232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d88830-bf58-45a5-a3d0-658e28a3e34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:02.130232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d88830-bf58-45a5-a3d0-658e28a3e34a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:02.130232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d88830-bf58-45a5-a3d0-658e28a3e34a admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:02.130232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d88830-bf58-45a5-a3d0-658e28a3e34a admin admin] PATH_INFO: `/v3/users/a90b21de083044f8b8626f403cd9bfb5` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:02.130232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d88830-bf58-45a5-a3d0-658e28a3e34a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a90b21de083044f8b8626f403cd9bfb5)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:02.131906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d88830-bf58-45a5-a3d0-658e28a3e34a admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:02.192619 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '699ef780b54c47e393b5a6dac5b33298', 'request_id': 'req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:02.198660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '699ef780b54c47e393b5a6dac5b33298', 'request_id': 'req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:02.204439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '699ef780b54c47e393b5a6dac5b33298', 'request_id': 'req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:02.424190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93 admin admin] The token cache is being invalidate because project 699ef780b54c47e393b5a6dac5b33298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:02.424190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:02.424190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:02.424190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1024/2066] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:01 2022] DELETE /identity/v3/projects/699ef780b54c47e393b5a6dac5b33298 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:02.424190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:02.424660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:02.424660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:02.424660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc621a-f0b3-4f7d-aa7f-9936417ded93 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:02.424873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07459bb8-4d5e-4008-a050-d3ee39ffd476 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5376633f469743c9b33076301c387120', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5ac1a712d7b476d831a4f579481b3dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:02.424873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07459bb8-4d5e-4008-a050-d3ee39ffd476 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:02.424873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07459bb8-4d5e-4008-a050-d3ee39ffd476 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:02.425100 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07459bb8-4d5e-4008-a050-d3ee39ffd476 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:02.425100 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07459bb8-4d5e-4008-a050-d3ee39ffd476 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:02.425100 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07459bb8-4d5e-4008-a050-d3ee39ffd476 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:02.425100 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07459bb8-4d5e-4008-a050-d3ee39ffd476 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:02.425100 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1025/2067] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:58:02.425429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e46eb-7ce7-43c9-b731-db5a32a876e7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:02.425429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e46eb-7ce7-43c9-b731-db5a32a876e7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:02.425429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e46eb-7ce7-43c9-b731-db5a32a876e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:02.425429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3b7e46eb-7ce7-43c9-b731-db5a32a876e7 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:02.425429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3b7e46eb-7ce7-43c9-b731-db5a32a876e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:02.434387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:02.440792 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-d0d88830-bf58-45a5-a3d0-658e28a3e34a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a90b21de083044f8b8626f403cd9bfb5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:02.447847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-d0d88830-bf58-45a5-a3d0-658e28a3e34a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a90b21de083044f8b8626f403cd9bfb5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:02.453305 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1026/2068] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:02.454151 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e46eb-7ce7-43c9-b731-db5a32a876e7 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:02.466402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-d0d88830-bf58-45a5-a3d0-658e28a3e34a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a90b21de083044f8b8626f403cd9bfb5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:02.467150 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-d0d88830-bf58-45a5-a3d0-658e28a3e34a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a90b21de083044f8b8626f403cd9bfb5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:02.467353 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e46eb-7ce7-43c9-b731-db5a32a876e7 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:02.467756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-d0d88830-bf58-45a5-a3d0-658e28a3e34a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a90b21de083044f8b8626f403cd9bfb5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:02.471981 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e9cd4-be3e-4d1c-86ac-a9158ec21949 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5376633f469743c9b33076301c387120', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5ac1a712d7b476d831a4f579481b3dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:02.475401 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e9cd4-be3e-4d1c-86ac-a9158ec21949 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:02.475401 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e9cd4-be3e-4d1c-86ac-a9158ec21949 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:02.475401 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e9cd4-be3e-4d1c-86ac-a9158ec21949 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:02.475401 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e9cd4-be3e-4d1c-86ac-a9158ec21949 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:02.476291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e9cd4-be3e-4d1c-86ac-a9158ec21949 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:02.586370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1043/2069] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:02 2022] DELETE /identity/v3/users/a90b21de083044f8b8626f403cd9bfb5 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:02.589586 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bbbdc-4d8f-4184-8784-d1f7cb94f186 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:02.589586 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bbbdc-4d8f-4184-8784-d1f7cb94f186 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:02.589586 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bbbdc-4d8f-4184-8784-d1f7cb94f186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:02.614044 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1027/2070] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:02 2022] POST /identity/v3/projects => generated 353 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:02.615886 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-0e8bbbdc-4d8f-4184-8784-d1f7cb94f186 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:02.617213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0e8bbbdc-4d8f-4184-8784-d1f7cb94f186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:02.617754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e9cd4-be3e-4d1c-86ac-a9158ec21949 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:02.620681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ff4e9cd4-be3e-4d1c-86ac-a9158ec21949 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:02.702454 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1044/2071] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:02.708975 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bbbdc-4d8f-4184-8784-d1f7cb94f186 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:02.712414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e9cd4-be3e-4d1c-86ac-a9158ec21949 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:02.717190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2ea95-acb9-4839-8da2-024c47e7cfd6 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5376633f469743c9b33076301c387120', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5ac1a712d7b476d831a4f579481b3dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:02.717800 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2ea95-acb9-4839-8da2-024c47e7cfd6 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:02.718146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2ea95-acb9-4839-8da2-024c47e7cfd6 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:02.718439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2ea95-acb9-4839-8da2-024c47e7cfd6 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:02.718543 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bbbdc-4d8f-4184-8784-d1f7cb94f186 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:02.719095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2ea95-acb9-4839-8da2-024c47e7cfd6 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:02.719445 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2ea95-acb9-4839-8da2-024c47e7cfd6 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:02.721039 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2ea95-acb9-4839-8da2-024c47e7cfd6 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:02.723316 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5992f18-7688-4ae5-a40c-f1e328155b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:02.723914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5992f18-7688-4ae5-a40c-f1e328155b35 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:02.724221 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5992f18-7688-4ae5-a40c-f1e328155b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:02.724495 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5992f18-7688-4ae5-a40c-f1e328155b35 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:02.725759 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5992f18-7688-4ae5-a40c-f1e328155b35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:02.726063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5992f18-7688-4ae5-a40c-f1e328155b35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:02.727736 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5992f18-7688-4ae5-a40c-f1e328155b35 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:02.733985 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1028/2072] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:58:02.742713 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1045/2073] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:58:02.744249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2ea95-acb9-4839-8da2-024c47e7cfd6 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:02.755567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2ea95-acb9-4839-8da2-024c47e7cfd6 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:02.760406 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d78fe-dd79-4f37-973e-a61ef1b6a4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:02.761071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d78fe-dd79-4f37-973e-a61ef1b6a4be admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:02.761420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d78fe-dd79-4f37-973e-a61ef1b6a4be admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:02.761754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d78fe-dd79-4f37-973e-a61ef1b6a4be admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:02.762648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d78fe-dd79-4f37-973e-a61ef1b6a4be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:02.764862 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d78fe-dd79-4f37-973e-a61ef1b6a4be admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:02.772598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5992f18-7688-4ae5-a40c-f1e328155b35 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:02.788081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5992f18-7688-4ae5-a40c-f1e328155b35 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:02.800236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e04734-92d5-47b6-815c-5135d6714f08 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5376633f469743c9b33076301c387120', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5ac1a712d7b476d831a4f579481b3dc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:02.800236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e04734-92d5-47b6-815c-5135d6714f08 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:02.800236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e04734-92d5-47b6-815c-5135d6714f08 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:02.800561 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e04734-92d5-47b6-815c-5135d6714f08 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] PATH_INFO: `/v3/projects/cc59b1a80f524a4286e6061581393277` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:02.800561 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e04734-92d5-47b6-815c-5135d6714f08 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cc59b1a80f524a4286e6061581393277)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:02.800561 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e04734-92d5-47b6-815c-5135d6714f08 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:02.887504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1029/2074] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:02 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:02.888587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d78fe-dd79-4f37-973e-a61ef1b6a4be admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:02.952376 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d78fe-dd79-4f37-973e-a61ef1b6a4be admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:02.954829 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-d1e04734-92d5-47b6-815c-5135d6714f08 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc59b1a80f524a4286e6061581393277', 'request_id': 'req-d1e04734-92d5-47b6-815c-5135d6714f08'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:02.958925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:02.962748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-d1e04734-92d5-47b6-815c-5135d6714f08 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc59b1a80f524a4286e6061581393277', 'request_id': 'req-d1e04734-92d5-47b6-815c-5135d6714f08'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:03.009736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b945457-43cc-4d70-a73a-bde85ba616af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:03.010254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b945457-43cc-4d70-a73a-bde85ba616af admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.010529 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b945457-43cc-4d70-a73a-bde85ba616af admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.010806 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b945457-43cc-4d70-a73a-bde85ba616af admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.011470 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b945457-43cc-4d70-a73a-bde85ba616af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:03.013392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b945457-43cc-4d70-a73a-bde85ba616af admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:03.049467 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-d1e04734-92d5-47b6-815c-5135d6714f08 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc59b1a80f524a4286e6061581393277', 'request_id': 'req-d1e04734-92d5-47b6-815c-5135d6714f08'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:03.069915 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-d1e04734-92d5-47b6-815c-5135d6714f08 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] The token cache is being invalidate because project cc59b1a80f524a4286e6061581393277 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:03.070198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-d1e04734-92d5-47b6-815c-5135d6714f08 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:03.071161 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-d1e04734-92d5-47b6-815c-5135d6714f08 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:03.072663 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1046/2075] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:02 2022] DELETE /identity/v3/projects/cc59b1a80f524a4286e6061581393277 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:03.073490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e04734-92d5-47b6-815c-5135d6714f08 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:03.074750 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d1e04734-92d5-47b6-815c-5135d6714f08 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:03.088913 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e04734-92d5-47b6-815c-5135d6714f08 tempest-VolumeQuotaClassesTest-2094553247 tempest-VolumeQuotaClassesTest-2094553247-project-admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:03.093711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e987a8a-1afd-4fdf-a014-4e9a23382697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:03.094321 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e987a8a-1afd-4fdf-a014-4e9a23382697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.094596 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e987a8a-1afd-4fdf-a014-4e9a23382697 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.094864 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e987a8a-1afd-4fdf-a014-4e9a23382697 admin admin] PATH_INFO: `/v3/projects/18f367dfc9764367b6e9dfe49082ac0a` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.096063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e987a8a-1afd-4fdf-a014-4e9a23382697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18f367dfc9764367b6e9dfe49082ac0a)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:03.098060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e987a8a-1afd-4fdf-a014-4e9a23382697 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:03.108380 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1030/2076] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:02 2022] POST /identity/v3/users => generated 287 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:03.113400 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b945457-43cc-4d70-a73a-bde85ba616af admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:03.114014 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3b945457-43cc-4d70-a73a-bde85ba616af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:03.126074 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b945457-43cc-4d70-a73a-bde85ba616af admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:03.126599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:03.130876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787b917-1c74-4c3e-bba2-4ef346acfe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:03.131906 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787b917-1c74-4c3e-bba2-4ef346acfe84 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.132207 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787b917-1c74-4c3e-bba2-4ef346acfe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.132794 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787b917-1c74-4c3e-bba2-4ef346acfe84 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.134116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787b917-1c74-4c3e-bba2-4ef346acfe84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:03.135854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787b917-1c74-4c3e-bba2-4ef346acfe84 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:03.145667 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1031/2077] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:03.149680 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787b917-1c74-4c3e-bba2-4ef346acfe84 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:03.159174 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787b917-1c74-4c3e-bba2-4ef346acfe84 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:03.164272 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c744a17-f11d-414d-b03b-99d1f918f496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:03.164842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c744a17-f11d-414d-b03b-99d1f918f496 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.165120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c744a17-f11d-414d-b03b-99d1f918f496 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.165390 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c744a17-f11d-414d-b03b-99d1f918f496 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.166014 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c744a17-f11d-414d-b03b-99d1f918f496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:03.167619 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c744a17-f11d-414d-b03b-99d1f918f496 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:03.177727 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1032/2078] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:03.182089 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c744a17-f11d-414d-b03b-99d1f918f496 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:03.191191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c744a17-f11d-414d-b03b-99d1f918f496 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:03.195890 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048f932-5137-4aa5-bcca-dea76c27fcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:03.196488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048f932-5137-4aa5-bcca-dea76c27fcf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.196765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048f932-5137-4aa5-bcca-dea76c27fcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.197035 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048f932-5137-4aa5-bcca-dea76c27fcf5 admin admin] PATH_INFO: `/v3/projects/43041628dfb44e9a86978ce90e3dd746/users/e74b1306b50844ec9c59df7625a50294/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.211168 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048f932-5137-4aa5-bcca-dea76c27fcf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43041628dfb44e9a86978ce90e3dd746, user_id=e74b1306b50844ec9c59df7625a50294, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:03.212233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1e987a8a-1afd-4fdf-a014-4e9a23382697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18f367dfc9764367b6e9dfe49082ac0a', 'request_id': 'req-1e987a8a-1afd-4fdf-a014-4e9a23382697'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:03.213906 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048f932-5137-4aa5-bcca-dea76c27fcf5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:03.217637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1e987a8a-1afd-4fdf-a014-4e9a23382697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18f367dfc9764367b6e9dfe49082ac0a', 'request_id': 'req-1e987a8a-1afd-4fdf-a014-4e9a23382697'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:03.293108 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1033/2079] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:03 2022] PUT /identity/v3/projects/43041628dfb44e9a86978ce90e3dd746/users/e74b1306b50844ec9c59df7625a50294/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:03.303414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a3311-9e4a-40f3-9cde-5d374d49720b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.303697 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a3311-9e4a-40f3-9cde-5d374d49720b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.303972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a3311-9e4a-40f3-9cde-5d374d49720b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.317287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1e987a8a-1afd-4fdf-a014-4e9a23382697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18f367dfc9764367b6e9dfe49082ac0a', 'request_id': 'req-1e987a8a-1afd-4fdf-a014-4e9a23382697'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:03.329721 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1e987a8a-1afd-4fdf-a014-4e9a23382697 admin admin] The token cache is being invalidate because project 18f367dfc9764367b6e9dfe49082ac0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:03.330006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1e987a8a-1afd-4fdf-a014-4e9a23382697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:03.330787 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1e987a8a-1afd-4fdf-a014-4e9a23382697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:03.332318 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1047/2080] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:03 2022] DELETE /identity/v3/projects/18f367dfc9764367b6e9dfe49082ac0a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:03.336262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c80a3311-9e4a-40f3-9cde-5d374d49720b None None] MFA Rules not processed for user `e74b1306b50844ec9c59df7625a50294`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:03.336262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c80a3311-9e4a-40f3-9cde-5d374d49720b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:03.337604 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b25b7-6755-45b0-aa16-ebe2b77331ce None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.337884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b25b7-6755-45b0-aa16-ebe2b77331ce None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.338370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b25b7-6755-45b0-aa16-ebe2b77331ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.358756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-aa9b25b7-6755-45b0-aa16-ebe2b77331ce None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:03.360009 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-aa9b25b7-6755-45b0-aa16-ebe2b77331ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:03.401727 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1034/2081] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:03.406275 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a3311-9e4a-40f3-9cde-5d374d49720b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:03.407728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c80a3311-9e4a-40f3-9cde-5d374d49720b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:03.420863 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:03.421763 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1048/2082] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:03.430404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79127f8b-6283-4b09-b610-17f28de19381 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.430679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79127f8b-6283-4b09-b610-17f28de19381 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.430952 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79127f8b-6283-4b09-b610-17f28de19381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.450177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-79127f8b-6283-4b09-b610-17f28de19381 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:03.451461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-79127f8b-6283-4b09-b610-17f28de19381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:03.459554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1049/2083] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:03.464116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79127f8b-6283-4b09-b610-17f28de19381 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:03.473374 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79127f8b-6283-4b09-b610-17f28de19381 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:03.485199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017fec3-aaa5-4dd0-a643-fe98b8f59086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:03.485331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a3311-9e4a-40f3-9cde-5d374d49720b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:03.485441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017fec3-aaa5-4dd0-a643-fe98b8f59086 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.485441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017fec3-aaa5-4dd0-a643-fe98b8f59086 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.485441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017fec3-aaa5-4dd0-a643-fe98b8f59086 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.485441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b017fec3-aaa5-4dd0-a643-fe98b8f59086 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:03.485441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b017fec3-aaa5-4dd0-a643-fe98b8f59086 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:03.485441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b017fec3-aaa5-4dd0-a643-fe98b8f59086 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:03.485948 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d34a6a-ad12-4a76-a857-52c6fa3e7746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:03.485948 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d34a6a-ad12-4a76-a857-52c6fa3e7746 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.485948 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d34a6a-ad12-4a76-a857-52c6fa3e7746 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.485948 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d34a6a-ad12-4a76-a857-52c6fa3e7746 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.486285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d34a6a-ad12-4a76-a857-52c6fa3e7746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:03.488068 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d34a6a-ad12-4a76-a857-52c6fa3e7746 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:03.496851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1050/2084] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:58:03.501145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017fec3-aaa5-4dd0-a643-fe98b8f59086 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:03.510706 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017fec3-aaa5-4dd0-a643-fe98b8f59086 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:03.515711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58845aa2-b2ea-4b4a-ade5-7d3803c16518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:03.515989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58845aa2-b2ea-4b4a-ade5-7d3803c16518 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.516172 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58845aa2-b2ea-4b4a-ade5-7d3803c16518 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.516466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58845aa2-b2ea-4b4a-ade5-7d3803c16518 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.516999 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58845aa2-b2ea-4b4a-ade5-7d3803c16518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:03.530256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58845aa2-b2ea-4b4a-ade5-7d3803c16518 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:03.613404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1035/2085] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:03 2022] POST /identity/v3/projects => generated 371 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:03.617482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d34a6a-ad12-4a76-a857-52c6fa3e7746 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:03.620756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1051/2086] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:03 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:03.621820 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58845aa2-b2ea-4b4a-ade5-7d3803c16518 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:03.688321 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d34a6a-ad12-4a76-a857-52c6fa3e7746 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:03.688566 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58845aa2-b2ea-4b4a-ade5-7d3803c16518 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:03.698424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6b4e7-1c58-472b-8be5-1c16783c4c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:03.698424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6b4e7-1c58-472b-8be5-1c16783c4c6d admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.698657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cce21d-99d1-4f95-8f3a-706f32eb8a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:03.698769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6b4e7-1c58-472b-8be5-1c16783c4c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.698769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6b4e7-1c58-472b-8be5-1c16783c4c6d admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.698938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cce21d-99d1-4f95-8f3a-706f32eb8a18 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.698938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cce21d-99d1-4f95-8f3a-706f32eb8a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.699138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6b4e7-1c58-472b-8be5-1c16783c4c6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:03.699236 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cce21d-99d1-4f95-8f3a-706f32eb8a18 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.699773 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cce21d-99d1-4f95-8f3a-706f32eb8a18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:03.700974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6b4e7-1c58-472b-8be5-1c16783c4c6d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:03.701734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cce21d-99d1-4f95-8f3a-706f32eb8a18 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:03.780782 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:03.786622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1036/2087] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:03 2022] POST /identity/v3/users => generated 292 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:03.791018 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1052/2088] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:03 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:03.791018 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6b4e7-1c58-472b-8be5-1c16783c4c6d admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:03.791285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cce21d-99d1-4f95-8f3a-706f32eb8a18 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:03.792702 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-24cce21d-99d1-4f95-8f3a-706f32eb8a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:03.801656 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6b4e7-1c58-472b-8be5-1c16783c4c6d admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:03.806745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cce21d-99d1-4f95-8f3a-706f32eb8a18 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:03.806917 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1e250-e918-4c9c-be94-4e9589461295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:03.807605 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1e250-e918-4c9c-be94-4e9589461295 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.807954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1e250-e918-4c9c-be94-4e9589461295 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.808334 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1e250-e918-4c9c-be94-4e9589461295 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.809033 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1e250-e918-4c9c-be94-4e9589461295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:03.810634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1e250-e918-4c9c-be94-4e9589461295 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:03.812696 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2f4d0-0572-4c17-928a-6a55689ed81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:03.816364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2f4d0-0572-4c17-928a-6a55689ed81c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.816364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2f4d0-0572-4c17-928a-6a55689ed81c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.816364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2f4d0-0572-4c17-928a-6a55689ed81c admin admin] PATH_INFO: `/v3/users/eb26cd4b255641dbaa7eb4d8ba9ebdd4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.816364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2f4d0-0572-4c17-928a-6a55689ed81c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb26cd4b255641dbaa7eb4d8ba9ebdd4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:03.817134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2f4d0-0572-4c17-928a-6a55689ed81c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:03.822628 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1053/2089] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:03.823595 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1e250-e918-4c9c-be94-4e9589461295 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:03.833249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1e250-e918-4c9c-be94-4e9589461295 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:03.838862 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689d40e-2c73-4008-b8c1-8d3568665c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:03.839500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689d40e-2c73-4008-b8c1-8d3568665c75 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.839962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689d40e-2c73-4008-b8c1-8d3568665c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.840251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689d40e-2c73-4008-b8c1-8d3568665c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.840898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689d40e-2c73-4008-b8c1-8d3568665c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:03.842635 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689d40e-2c73-4008-b8c1-8d3568665c75 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:03.853769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1054/2090] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:03.854590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689d40e-2c73-4008-b8c1-8d3568665c75 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:03.864821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689d40e-2c73-4008-b8c1-8d3568665c75 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:03.869559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a68e6-5a02-4acf-96a0-e6aa5793c48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:03.874465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a68e6-5a02-4acf-96a0-e6aa5793c48a admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:03.874465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a68e6-5a02-4acf-96a0-e6aa5793c48a admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:03.874465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a68e6-5a02-4acf-96a0-e6aa5793c48a admin admin] PATH_INFO: `/v3/projects/2a39c8a4f22f46b2be1b2827be7be619/users/12db4ced48864d1399412563d86b44fc/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:03.887662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a68e6-5a02-4acf-96a0-e6aa5793c48a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a39c8a4f22f46b2be1b2827be7be619, user_id=12db4ced48864d1399412563d86b44fc, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:03.890315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a68e6-5a02-4acf-96a0-e6aa5793c48a admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:04.041734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1055/2091] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:03 2022] PUT /identity/v3/projects/2a39c8a4f22f46b2be1b2827be7be619/users/12db4ced48864d1399412563d86b44fc/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:04.043273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a68e6-5a02-4acf-96a0-e6aa5793c48a admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:04.079444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b3a2f4d0-0572-4c17-928a-6a55689ed81c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb26cd4b255641dbaa7eb4d8ba9ebdd4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:04.087252 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b3a2f4d0-0572-4c17-928a-6a55689ed81c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb26cd4b255641dbaa7eb4d8ba9ebdd4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:04.088001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b3a2f4d0-0572-4c17-928a-6a55689ed81c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb26cd4b255641dbaa7eb4d8ba9ebdd4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:04.106432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a68e6-5a02-4acf-96a0-e6aa5793c48a admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:04.111214 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba10922-6f5f-448d-a577-f291f5b61c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:04.111805 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba10922-6f5f-448d-a577-f291f5b61c9e admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:04.112078 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba10922-6f5f-448d-a577-f291f5b61c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:04.112346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba10922-6f5f-448d-a577-f291f5b61c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:04.112977 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba10922-6f5f-448d-a577-f291f5b61c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:04.114643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba10922-6f5f-448d-a577-f291f5b61c9e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:04.128193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1056/2092] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:04.129141 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba10922-6f5f-448d-a577-f291f5b61c9e admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:04.130854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9ba10922-6f5f-448d-a577-f291f5b61c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:04.143454 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba10922-6f5f-448d-a577-f291f5b61c9e admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:04.149256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea086db-3dab-46a8-b66f-c927affb5e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:04.149828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea086db-3dab-46a8-b66f-c927affb5e10 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:04.150099 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea086db-3dab-46a8-b66f-c927affb5e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:04.150390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea086db-3dab-46a8-b66f-c927affb5e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:04.150984 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea086db-3dab-46a8-b66f-c927affb5e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:04.152578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea086db-3dab-46a8-b66f-c927affb5e10 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:04.162176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1057/2093] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:04.162988 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea086db-3dab-46a8-b66f-c927affb5e10 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:04.167051 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-3ea086db-3dab-46a8-b66f-c927affb5e10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:04.179362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea086db-3dab-46a8-b66f-c927affb5e10 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:04.183233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b3a2f4d0-0572-4c17-928a-6a55689ed81c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb26cd4b255641dbaa7eb4d8ba9ebdd4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:04.184121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a95a4-1acc-4088-9029-01bf1f61ef5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:04.184744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a95a4-1acc-4088-9029-01bf1f61ef5b admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:04.185021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a95a4-1acc-4088-9029-01bf1f61ef5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:04.185287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a95a4-1acc-4088-9029-01bf1f61ef5b admin admin] PATH_INFO: `/v3/projects/ba5604a408ba402c9a0bfe2b2d78955b/users/66cd4f38ca4e4826a3a0d108a840530e/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:04.193018 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b3a2f4d0-0572-4c17-928a-6a55689ed81c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb26cd4b255641dbaa7eb4d8ba9ebdd4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:04.194539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:04.199229 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1037/2094] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:03 2022] DELETE /identity/v3/users/eb26cd4b255641dbaa7eb4d8ba9ebdd4 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:04.199440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a95a4-1acc-4088-9029-01bf1f61ef5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba5604a408ba402c9a0bfe2b2d78955b, user_id=66cd4f38ca4e4826a3a0d108a840530e, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:04.199992 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2f4d0-0572-4c17-928a-6a55689ed81c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:04.202214 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a95a4-1acc-4088-9029-01bf1f61ef5b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:04.260726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2f4d0-0572-4c17-928a-6a55689ed81c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:04.265736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf0aa3-7b8d-440c-881e-057f17a523cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:04.266398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf0aa3-7b8d-440c-881e-057f17a523cb admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:04.266678 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf0aa3-7b8d-440c-881e-057f17a523cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:04.266952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf0aa3-7b8d-440c-881e-057f17a523cb admin admin] PATH_INFO: `/v3/projects/2a39c8a4f22f46b2be1b2827be7be619/users/12db4ced48864d1399412563d86b44fc/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:04.269008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf0aa3-7b8d-440c-881e-057f17a523cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a39c8a4f22f46b2be1b2827be7be619, user_id=12db4ced48864d1399412563d86b44fc, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:04.271692 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf0aa3-7b8d-440c-881e-057f17a523cb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:04.304295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1038/2095] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:04 2022] PUT /identity/v3/projects/2a39c8a4f22f46b2be1b2827be7be619/users/12db4ced48864d1399412563d86b44fc/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:04.304295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b50d2-09cf-4ce3-ab32-1594cb16dcb9 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:04.304295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b50d2-09cf-4ce3-ab32-1594cb16dcb9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:04.304295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b50d2-09cf-4ce3-ab32-1594cb16dcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:04.304800 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1058/2096] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:04 2022] PUT /identity/v3/projects/ba5604a408ba402c9a0bfe2b2d78955b/users/66cd4f38ca4e4826a3a0d108a840530e/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:04.304800 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c123ed-65c7-4cbe-8460-714215502f56 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:04.304800 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c123ed-65c7-4cbe-8460-714215502f56 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:04.304800 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c123ed-65c7-4cbe-8460-714215502f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:04.307674 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-be4b50d2-09cf-4ce3-ab32-1594cb16dcb9 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:04.309051 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-be4b50d2-09cf-4ce3-ab32-1594cb16dcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:04.329056 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-a8c123ed-65c7-4cbe-8460-714215502f56 None None] MFA Rules not processed for user `12db4ced48864d1399412563d86b44fc`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:04.330276 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a8c123ed-65c7-4cbe-8460-714215502f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:04.371639 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1039/2097] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:04.376038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a413-c056-4d73-9020-42551c274832 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:04.376326 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a413-c056-4d73-9020-42551c274832 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:04.376600 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a413-c056-4d73-9020-42551c274832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:04.396249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:04.435815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1059/2098] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:04.435815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c123ed-65c7-4cbe-8460-714215502f56 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:04.435815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a8c123ed-65c7-4cbe-8460-714215502f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:04.436163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-da45a413-c056-4d73-9020-42551c274832 None None] MFA Rules not processed for user `66cd4f38ca4e4826a3a0d108a840530e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:04.436163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-da45a413-c056-4d73-9020-42551c274832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:04.464908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c123ed-65c7-4cbe-8460-714215502f56 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:04.469055 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590cbe7-df1c-4348-a1cb-aebac75091f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:04.469655 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590cbe7-df1c-4348-a1cb-aebac75091f4 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:04.469927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590cbe7-df1c-4348-a1cb-aebac75091f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:04.470195 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590cbe7-df1c-4348-a1cb-aebac75091f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:04.472512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1040/2099] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:04.473320 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45a413-c056-4d73-9020-42551c274832 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:04.474560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-da45a413-c056-4d73-9020-42551c274832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:04.480335 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590cbe7-df1c-4348-a1cb-aebac75091f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:04.482151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590cbe7-df1c-4348-a1cb-aebac75091f4 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:04.492438 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1060/2100] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:04.493369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590cbe7-df1c-4348-a1cb-aebac75091f4 service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:04.494730 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c590cbe7-df1c-4348-a1cb-aebac75091f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:04.535559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45a413-c056-4d73-9020-42551c274832 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:04.539832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad5b8e-f5cc-401c-b7b2-b49683569879 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:04.540470 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad5b8e-f5cc-401c-b7b2-b49683569879 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:04.540746 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad5b8e-f5cc-401c-b7b2-b49683569879 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:04.541017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad5b8e-f5cc-401c-b7b2-b49683569879 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:04.552390 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad5b8e-f5cc-401c-b7b2-b49683569879 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:04.553933 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad5b8e-f5cc-401c-b7b2-b49683569879 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:04.559855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590cbe7-df1c-4348-a1cb-aebac75091f4 service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:04.563023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070000b2-a30c-4c82-b0cc-506a2bc28534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:04.563670 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070000b2-a30c-4c82-b0cc-506a2bc28534 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:04.563946 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070000b2-a30c-4c82-b0cc-506a2bc28534 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:04.564215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070000b2-a30c-4c82-b0cc-506a2bc28534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:04.564690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1041/2101] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:04.574165 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070000b2-a30c-4c82-b0cc-506a2bc28534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:04.575633 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070000b2-a30c-4c82-b0cc-506a2bc28534 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:04.590802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1061/2102] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:04.733359 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:04.986092 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad5b8e-f5cc-401c-b7b2-b49683569879 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:04.987608 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e2ad5b8e-f5cc-401c-b7b2-b49683569879 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:05.002797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad5b8e-f5cc-401c-b7b2-b49683569879 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:05.008671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e808f-a0b6-42d0-87b3-23b97043b064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:05.009722 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e808f-a0b6-42d0-87b3-23b97043b064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:05.010226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e808f-a0b6-42d0-87b3-23b97043b064 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:05.010644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e808f-a0b6-42d0-87b3-23b97043b064 admin admin] PATH_INFO: `/v3/projects/02703ffb2f964879b35bbe443a49e848` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:05.012134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e808f-a0b6-42d0-87b3-23b97043b064 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02703ffb2f964879b35bbe443a49e848)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:05.014942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e808f-a0b6-42d0-87b3-23b97043b064 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:05.196877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-be9e808f-a0b6-42d0-87b3-23b97043b064 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02703ffb2f964879b35bbe443a49e848', 'request_id': 'req-be9e808f-a0b6-42d0-87b3-23b97043b064'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:05.204539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-be9e808f-a0b6-42d0-87b3-23b97043b064 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02703ffb2f964879b35bbe443a49e848', 'request_id': 'req-be9e808f-a0b6-42d0-87b3-23b97043b064'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:05.216313 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-be9e808f-a0b6-42d0-87b3-23b97043b064 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02703ffb2f964879b35bbe443a49e848', 'request_id': 'req-be9e808f-a0b6-42d0-87b3-23b97043b064'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:05.299670 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-be9e808f-a0b6-42d0-87b3-23b97043b064 admin admin] The token cache is being invalidate because project 02703ffb2f964879b35bbe443a49e848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:05.300093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-be9e808f-a0b6-42d0-87b3-23b97043b064 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:05.300799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-be9e808f-a0b6-42d0-87b3-23b97043b064 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:05.302740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1042/2103] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:04 2022] DELETE /identity/v3/projects/02703ffb2f964879b35bbe443a49e848 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:05.307149 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070000b2-a30c-4c82-b0cc-506a2bc28534 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:05.321647 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070000b2-a30c-4c82-b0cc-506a2bc28534 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:05.385915 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1870783b-d353-4c31-a1da-b02e8329d022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:05.386576 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870783b-d353-4c31-a1da-b02e8329d022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:05.386857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870783b-d353-4c31-a1da-b02e8329d022 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:05.387148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870783b-d353-4c31-a1da-b02e8329d022 admin admin] PATH_INFO: `/v3/users/5376633f469743c9b33076301c387120` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:05.388294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870783b-d353-4c31-a1da-b02e8329d022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5376633f469743c9b33076301c387120)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:05.393680 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1870783b-d353-4c31-a1da-b02e8329d022 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:05.591156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:05.593018 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1870783b-d353-4c31-a1da-b02e8329d022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5376633f469743c9b33076301c387120'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:05.600209 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1870783b-d353-4c31-a1da-b02e8329d022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5376633f469743c9b33076301c387120'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:05.609559 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e808f-a0b6-42d0-87b3-23b97043b064 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:05.612573 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-be9e808f-a0b6-42d0-87b3-23b97043b064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:05.618169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1870783b-d353-4c31-a1da-b02e8329d022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5376633f469743c9b33076301c387120'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:05.618853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1870783b-d353-4c31-a1da-b02e8329d022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5376633f469743c9b33076301c387120'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:05.619463 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1870783b-d353-4c31-a1da-b02e8329d022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5376633f469743c9b33076301c387120'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:05.624488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e808f-a0b6-42d0-87b3-23b97043b064 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:05.628986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41ab91-5689-4083-813b-948d29563d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:05.629596 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41ab91-5689-4083-813b-948d29563d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:05.629927 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41ab91-5689-4083-813b-948d29563d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:05.630204 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41ab91-5689-4083-813b-948d29563d6e admin admin] PATH_INFO: `/v3/users/e74b1306b50844ec9c59df7625a50294` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:05.631345 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41ab91-5689-4083-813b-948d29563d6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e74b1306b50844ec9c59df7625a50294)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:05.633267 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41ab91-5689-4083-813b-948d29563d6e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:05.699978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1062/2104] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:05 2022] DELETE /identity/v3/users/5376633f469743c9b33076301c387120 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:05.828454 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5e41ab91-5689-4083-813b-948d29563d6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e74b1306b50844ec9c59df7625a50294'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:05.836158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5e41ab91-5689-4083-813b-948d29563d6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e74b1306b50844ec9c59df7625a50294'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:05.836851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5e41ab91-5689-4083-813b-948d29563d6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e74b1306b50844ec9c59df7625a50294'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:05.928839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5e41ab91-5689-4083-813b-948d29563d6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e74b1306b50844ec9c59df7625a50294'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:05.944438 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5e41ab91-5689-4083-813b-948d29563d6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e74b1306b50844ec9c59df7625a50294'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:05.951488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1043/2105] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:05 2022] DELETE /identity/v3/users/e74b1306b50844ec9c59df7625a50294 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:05.956276 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879307ca-3db4-493f-8bac-1a4d0966214c None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:05.956568 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879307ca-3db4-493f-8bac-1a4d0966214c None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:05.956871 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879307ca-3db4-493f-8bac-1a4d0966214c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:05.976924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-879307ca-3db4-493f-8bac-1a4d0966214c None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:05.978156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-879307ca-3db4-493f-8bac-1a4d0966214c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:06.039705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1063/2106] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:06.049446 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41ab91-5689-4083-813b-948d29563d6e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:06.052508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5e41ab91-5689-4083-813b-948d29563d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:06.074681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:06.110268 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41ab91-5689-4083-813b-948d29563d6e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:06.113884 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35207c7c-12bd-43d2-8599-42548cdb181a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:06.114482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35207c7c-12bd-43d2-8599-42548cdb181a service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:06.114780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35207c7c-12bd-43d2-8599-42548cdb181a service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:06.115026 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35207c7c-12bd-43d2-8599-42548cdb181a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:06.124377 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35207c7c-12bd-43d2-8599-42548cdb181a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:06.125828 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35207c7c-12bd-43d2-8599-42548cdb181a service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:06.136377 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1044/2107] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:06.144282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879307ca-3db4-493f-8bac-1a4d0966214c None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:06.145679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-879307ca-3db4-493f-8bac-1a4d0966214c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:06.158208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879307ca-3db4-493f-8bac-1a4d0966214c None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:06.163383 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661380f-006e-4965-bc1c-0b8f8c165f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:06.164217 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661380f-006e-4965-bc1c-0b8f8c165f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:06.164508 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661380f-006e-4965-bc1c-0b8f8c165f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:06.167901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661380f-006e-4965-bc1c-0b8f8c165f45 admin admin] PATH_INFO: `/v3/projects/a5ac1a712d7b476d831a4f579481b3dc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:06.167901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661380f-006e-4965-bc1c-0b8f8c165f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5ac1a712d7b476d831a4f579481b3dc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:06.168287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661380f-006e-4965-bc1c-0b8f8c165f45 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:06.289727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a661380f-006e-4965-bc1c-0b8f8c165f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5ac1a712d7b476d831a4f579481b3dc', 'request_id': 'req-a661380f-006e-4965-bc1c-0b8f8c165f45'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:06.295958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a661380f-006e-4965-bc1c-0b8f8c165f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5ac1a712d7b476d831a4f579481b3dc', 'request_id': 'req-a661380f-006e-4965-bc1c-0b8f8c165f45'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:06.380216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a661380f-006e-4965-bc1c-0b8f8c165f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5ac1a712d7b476d831a4f579481b3dc', 'request_id': 'req-a661380f-006e-4965-bc1c-0b8f8c165f45'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:06.395723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a661380f-006e-4965-bc1c-0b8f8c165f45 admin admin] The token cache is being invalidate because project a5ac1a712d7b476d831a4f579481b3dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:06.396195 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a661380f-006e-4965-bc1c-0b8f8c165f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:06.397027 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a661380f-006e-4965-bc1c-0b8f8c165f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:06.398887 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1064/2108] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:06 2022] DELETE /identity/v3/projects/a5ac1a712d7b476d831a4f579481b3dc => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:06.403963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae70d59-37f2-4131-a2c5-2be525d95d6c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:06.404419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae70d59-37f2-4131-a2c5-2be525d95d6c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:06.410691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae70d59-37f2-4131-a2c5-2be525d95d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:06.436347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-fae70d59-37f2-4131-a2c5-2be525d95d6c None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:06.436347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fae70d59-37f2-4131-a2c5-2be525d95d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:06.503805 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1045/2109] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:06.512378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c57e1-ca62-476c-b56e-4929358adb72 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:06.512658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c57e1-ca62-476c-b56e-4929358adb72 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:06.512931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c57e1-ca62-476c-b56e-4929358adb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:06.534476 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-761c57e1-ca62-476c-b56e-4929358adb72 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:06.535996 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-761c57e1-ca62-476c-b56e-4929358adb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:06.544156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1065/2110] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:06.548452 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae70d59-37f2-4131-a2c5-2be525d95d6c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:06.558527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae70d59-37f2-4131-a2c5-2be525d95d6c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:06.563220 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac7ed9-92ff-4113-aec1-d06667ab8329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:06.563815 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac7ed9-92ff-4113-aec1-d06667ab8329 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:06.567998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac7ed9-92ff-4113-aec1-d06667ab8329 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:06.567998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac7ed9-92ff-4113-aec1-d06667ab8329 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:06.567998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac7ed9-92ff-4113-aec1-d06667ab8329 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:06.567998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac7ed9-92ff-4113-aec1-d06667ab8329 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:06.567998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac7ed9-92ff-4113-aec1-d06667ab8329 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:06.579297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1046/2111] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:58:06.583767 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c57e1-ca62-476c-b56e-4929358adb72 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:06.593774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c57e1-ca62-476c-b56e-4929358adb72 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:06.598563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93bf2b-755d-4698-8ee2-dbe26b48393b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:06.599178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93bf2b-755d-4698-8ee2-dbe26b48393b admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:06.599510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93bf2b-755d-4698-8ee2-dbe26b48393b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:06.599792 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93bf2b-755d-4698-8ee2-dbe26b48393b admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:06.600472 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93bf2b-755d-4698-8ee2-dbe26b48393b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:06.602397 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93bf2b-755d-4698-8ee2-dbe26b48393b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:06.661311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac7ed9-92ff-4113-aec1-d06667ab8329 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:06.663226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b0ac7ed9-92ff-4113-aec1-d06667ab8329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:06.683564 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac7ed9-92ff-4113-aec1-d06667ab8329 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:06.690459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1066/2112] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:06 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:06.691321 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93bf2b-755d-4698-8ee2-dbe26b48393b admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:06.712964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:06.748777 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed35d4-5023-4366-ba39-749c7ee43c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:06.749408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed35d4-5023-4366-ba39-749c7ee43c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:06.749687 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed35d4-5023-4366-ba39-749c7ee43c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:06.749959 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed35d4-5023-4366-ba39-749c7ee43c9c admin admin] PATH_INFO: `/v3/projects/43041628dfb44e9a86978ce90e3dd746` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:06.751067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed35d4-5023-4366-ba39-749c7ee43c9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=43041628dfb44e9a86978ce90e3dd746)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:06.753013 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed35d4-5023-4366-ba39-749c7ee43c9c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:06.757225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93bf2b-755d-4698-8ee2-dbe26b48393b admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:06.762047 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bea7a8-b423-40cf-b483-3d6fee5de290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:06.764297 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bea7a8-b423-40cf-b483-3d6fee5de290 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:06.764297 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bea7a8-b423-40cf-b483-3d6fee5de290 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:06.764297 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bea7a8-b423-40cf-b483-3d6fee5de290 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:06.764297 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bea7a8-b423-40cf-b483-3d6fee5de290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:06.765779 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bea7a8-b423-40cf-b483-3d6fee5de290 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:06.875503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1067/2113] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:06 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:06.879930 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bea7a8-b423-40cf-b483-3d6fee5de290 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:06.889610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bea7a8-b423-40cf-b483-3d6fee5de290 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:06.894997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b78003-9832-41cb-ab85-d51aa91b457d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:06.895610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b78003-9832-41cb-ab85-d51aa91b457d admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:06.895885 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b78003-9832-41cb-ab85-d51aa91b457d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:06.896152 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b78003-9832-41cb-ab85-d51aa91b457d admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:06.896866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b78003-9832-41cb-ab85-d51aa91b457d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:06.898589 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b78003-9832-41cb-ab85-d51aa91b457d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:06.909203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1068/2114] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:06.914112 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b78003-9832-41cb-ab85-d51aa91b457d admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:06.933725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b78003-9832-41cb-ab85-d51aa91b457d admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:06.938357 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-71ed35d4-5023-4366-ba39-749c7ee43c9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43041628dfb44e9a86978ce90e3dd746', 'request_id': 'req-71ed35d4-5023-4366-ba39-749c7ee43c9c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:06.947564 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-71ed35d4-5023-4366-ba39-749c7ee43c9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43041628dfb44e9a86978ce90e3dd746', 'request_id': 'req-71ed35d4-5023-4366-ba39-749c7ee43c9c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:06.954169 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-71ed35d4-5023-4366-ba39-749c7ee43c9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43041628dfb44e9a86978ce90e3dd746', 'request_id': 'req-71ed35d4-5023-4366-ba39-749c7ee43c9c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:06.998061 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bced729-1b7a-40aa-95c9-6161cf44e915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:06.998654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bced729-1b7a-40aa-95c9-6161cf44e915 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:06.998928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bced729-1b7a-40aa-95c9-6161cf44e915 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:06.999258 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bced729-1b7a-40aa-95c9-6161cf44e915 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:06.999942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bced729-1b7a-40aa-95c9-6161cf44e915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:07.001557 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bced729-1b7a-40aa-95c9-6161cf44e915 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:07.011763 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1069/2115] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:07.016159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bced729-1b7a-40aa-95c9-6161cf44e915 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:07.026491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bced729-1b7a-40aa-95c9-6161cf44e915 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:07.031198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27baf231-78e2-4802-af80-d1fe18753e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:07.031831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27baf231-78e2-4802-af80-d1fe18753e3c admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:07.032108 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27baf231-78e2-4802-af80-d1fe18753e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:07.032380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27baf231-78e2-4802-af80-d1fe18753e3c admin admin] PATH_INFO: `/v3/projects/87fdafb470e34d01bc58a49d69e3d97c/users/603e331893634640986a040ecb4dd64d/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:07.049454 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27baf231-78e2-4802-af80-d1fe18753e3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=87fdafb470e34d01bc58a49d69e3d97c, user_id=603e331893634640986a040ecb4dd64d, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:07.052457 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27baf231-78e2-4802-af80-d1fe18753e3c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:07.075224 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-71ed35d4-5023-4366-ba39-749c7ee43c9c admin admin] The token cache is being invalidate because project 43041628dfb44e9a86978ce90e3dd746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:07.077763 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-71ed35d4-5023-4366-ba39-749c7ee43c9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:07.077763 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-71ed35d4-5023-4366-ba39-749c7ee43c9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:07.077763 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1047/2116] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:06 2022] DELETE /identity/v3/projects/43041628dfb44e9a86978ce90e3dd746 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:07.081666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed35d4-5023-4366-ba39-749c7ee43c9c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:07.082957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-71ed35d4-5023-4366-ba39-749c7ee43c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:07.093269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:07.096826 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed35d4-5023-4366-ba39-749c7ee43c9c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:07.102101 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eff4ce-e5a7-4690-a752-a5895d2c1cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:07.102687 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eff4ce-e5a7-4690-a752-a5895d2c1cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:07.102972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eff4ce-e5a7-4690-a752-a5895d2c1cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:07.103276 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eff4ce-e5a7-4690-a752-a5895d2c1cd3 admin admin] PATH_INFO: `/v3/users/12db4ced48864d1399412563d86b44fc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:07.104394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eff4ce-e5a7-4690-a752-a5895d2c1cd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12db4ced48864d1399412563d86b44fc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:07.106561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eff4ce-e5a7-4690-a752-a5895d2c1cd3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:07.138465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1070/2117] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:07 2022] PUT /identity/v3/projects/87fdafb470e34d01bc58a49d69e3d97c/users/603e331893634640986a040ecb4dd64d/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:07.150195 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365a5ce-ddc7-4ec5-b065-81d6a0a79b6e None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:07.150473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365a5ce-ddc7-4ec5-b065-81d6a0a79b6e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:07.150738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365a5ce-ddc7-4ec5-b065-81d6a0a79b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:07.177729 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-8365a5ce-ddc7-4ec5-b065-81d6a0a79b6e None None] MFA Rules not processed for user `603e331893634640986a040ecb4dd64d`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:07.178911 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8365a5ce-ddc7-4ec5-b065-81d6a0a79b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:07.242633 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1071/2118] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:07.247128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365a5ce-ddc7-4ec5-b065-81d6a0a79b6e None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:07.248476 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8365a5ce-ddc7-4ec5-b065-81d6a0a79b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:07.268080 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-31eff4ce-e5a7-4690-a752-a5895d2c1cd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12db4ced48864d1399412563d86b44fc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:07.275736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-31eff4ce-e5a7-4690-a752-a5895d2c1cd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12db4ced48864d1399412563d86b44fc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:07.276423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-31eff4ce-e5a7-4690-a752-a5895d2c1cd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12db4ced48864d1399412563d86b44fc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:07.314526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365a5ce-ddc7-4ec5-b065-81d6a0a79b6e None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:07.319188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4944b-d93d-4ec8-9196-c3ed7aabc3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:07.322578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4944b-d93d-4ec8-9196-c3ed7aabc3e1 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:07.322578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4944b-d93d-4ec8-9196-c3ed7aabc3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:07.322578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4944b-d93d-4ec8-9196-c3ed7aabc3e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:07.322578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4944b-d93d-4ec8-9196-c3ed7aabc3e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:07.323155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4944b-d93d-4ec8-9196-c3ed7aabc3e1 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:07.374530 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-31eff4ce-e5a7-4690-a752-a5895d2c1cd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12db4ced48864d1399412563d86b44fc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:07.388000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-31eff4ce-e5a7-4690-a752-a5895d2c1cd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12db4ced48864d1399412563d86b44fc'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:07.393960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1048/2119] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:07 2022] DELETE /identity/v3/users/12db4ced48864d1399412563d86b44fc => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:07.471558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1072/2120] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:07 2022] POST /identity/v3/projects => generated 385 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:07.475646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eff4ce-e5a7-4690-a752-a5895d2c1cd3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:07.477072 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-31eff4ce-e5a7-4690-a752-a5895d2c1cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:07.540838 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eff4ce-e5a7-4690-a752-a5895d2c1cd3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:07.595517 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557818f-b7b7-4b31-b884-22ff4c4b1a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:07.596153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557818f-b7b7-4b31-b884-22ff4c4b1a79 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:07.596401 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557818f-b7b7-4b31-b884-22ff4c4b1a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:07.596675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557818f-b7b7-4b31-b884-22ff4c4b1a79 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:07.597300 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557818f-b7b7-4b31-b884-22ff4c4b1a79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:07.599274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557818f-b7b7-4b31-b884-22ff4c4b1a79 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:07.604380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:07.658847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:07.696540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1049/2121] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:07 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:07.697706 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4944b-d93d-4ec8-9196-c3ed7aabc3e1 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:07.707927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4944b-d93d-4ec8-9196-c3ed7aabc3e1 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:07.713083 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe8487-01cc-4b8b-9398-be0b59735f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:07.713727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe8487-01cc-4b8b-9398-be0b59735f49 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:07.714005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe8487-01cc-4b8b-9398-be0b59735f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:07.714273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe8487-01cc-4b8b-9398-be0b59735f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:07.715441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe8487-01cc-4b8b-9398-be0b59735f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:07.716679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe8487-01cc-4b8b-9398-be0b59735f49 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:07.727614 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1073/2122] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:07.731763 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557818f-b7b7-4b31-b884-22ff4c4b1a79 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:07.742744 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557818f-b7b7-4b31-b884-22ff4c4b1a79 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:07.747505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506aa5ca-1913-4164-8bc6-70f8d3653310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:07.748111 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506aa5ca-1913-4164-8bc6-70f8d3653310 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:07.748381 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506aa5ca-1913-4164-8bc6-70f8d3653310 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:07.748649 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506aa5ca-1913-4164-8bc6-70f8d3653310 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:07.749335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506aa5ca-1913-4164-8bc6-70f8d3653310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:07.750931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506aa5ca-1913-4164-8bc6-70f8d3653310 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:07.761870 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1050/2123] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:07.766004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe8487-01cc-4b8b-9398-be0b59735f49 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:07.775950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe8487-01cc-4b8b-9398-be0b59735f49 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:07.780534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd97b5-4633-4403-a44a-1c70cce2c0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:07.781177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd97b5-4633-4403-a44a-1c70cce2c0ff admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:07.781459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd97b5-4633-4403-a44a-1c70cce2c0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:07.781737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd97b5-4633-4403-a44a-1c70cce2c0ff admin admin] PATH_INFO: `/v3/projects/03ffcce6b6ef49b1a50a9f2ed99471e2/users/00989e8a404546e6a1b1e4db9e06744e/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:07.796173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd97b5-4633-4403-a44a-1c70cce2c0ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=03ffcce6b6ef49b1a50a9f2ed99471e2, user_id=00989e8a404546e6a1b1e4db9e06744e, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:07.798865 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd97b5-4633-4403-a44a-1c70cce2c0ff admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:07.820756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506aa5ca-1913-4164-8bc6-70f8d3653310 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:07.820756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-506aa5ca-1913-4164-8bc6-70f8d3653310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:07.831157 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506aa5ca-1913-4164-8bc6-70f8d3653310 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:07.837632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2e7a0-e4bb-4456-a4e9-303413b82421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:07.843848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2e7a0-e4bb-4456-a4e9-303413b82421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:07.843848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2e7a0-e4bb-4456-a4e9-303413b82421 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:07.843848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2e7a0-e4bb-4456-a4e9-303413b82421 admin admin] PATH_INFO: `/v3/projects/2a39c8a4f22f46b2be1b2827be7be619` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:07.843848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2e7a0-e4bb-4456-a4e9-303413b82421 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a39c8a4f22f46b2be1b2827be7be619)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:07.843848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2e7a0-e4bb-4456-a4e9-303413b82421 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:07.864885 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1074/2124] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:07 2022] PUT /identity/v3/projects/03ffcce6b6ef49b1a50a9f2ed99471e2/users/00989e8a404546e6a1b1e4db9e06744e/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:07.875570 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624666e1-7011-4406-8a94-820219d4b5da None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:07.875859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624666e1-7011-4406-8a94-820219d4b5da None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:07.876132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624666e1-7011-4406-8a94-820219d4b5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:07.903740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-624666e1-7011-4406-8a94-820219d4b5da None None] MFA Rules not processed for user `00989e8a404546e6a1b1e4db9e06744e`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:07.904979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-624666e1-7011-4406-8a94-820219d4b5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:07.976125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1075/2125] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:07.976125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624666e1-7011-4406-8a94-820219d4b5da None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:07.979617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-97f2e7a0-e4bb-4456-a4e9-303413b82421 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a39c8a4f22f46b2be1b2827be7be619', 'request_id': 'req-97f2e7a0-e4bb-4456-a4e9-303413b82421'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:07.987009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-97f2e7a0-e4bb-4456-a4e9-303413b82421 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a39c8a4f22f46b2be1b2827be7be619', 'request_id': 'req-97f2e7a0-e4bb-4456-a4e9-303413b82421'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:07.993550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-97f2e7a0-e4bb-4456-a4e9-303413b82421 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a39c8a4f22f46b2be1b2827be7be619', 'request_id': 'req-97f2e7a0-e4bb-4456-a4e9-303413b82421'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:08.022167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:08.042818 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624666e1-7011-4406-8a94-820219d4b5da None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:08.076931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-97f2e7a0-e4bb-4456-a4e9-303413b82421 admin admin] The token cache is being invalidate because project 2a39c8a4f22f46b2be1b2827be7be619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:08.077222 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-97f2e7a0-e4bb-4456-a4e9-303413b82421 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:08.077871 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-97f2e7a0-e4bb-4456-a4e9-303413b82421 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:08.079527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1051/2126] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:07 2022] DELETE /identity/v3/projects/2a39c8a4f22f46b2be1b2827be7be619 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:08.084217 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7d841-355a-456d-bafc-49985114d7b9 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.084501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7d841-355a-456d-bafc-49985114d7b9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.084775 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7d841-355a-456d-bafc-49985114d7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.099047 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d9ac9-a80b-4798-8b45-b150a3dc89e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:08.099678 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d9ac9-a80b-4798-8b45-b150a3dc89e2 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.099952 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d9ac9-a80b-4798-8b45-b150a3dc89e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.100221 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d9ac9-a80b-4798-8b45-b150a3dc89e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.100894 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d9ac9-a80b-4798-8b45-b150a3dc89e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:08.102787 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d9ac9-a80b-4798-8b45-b150a3dc89e2 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:08.103947 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-92c7d841-355a-456d-bafc-49985114d7b9 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:08.105155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-92c7d841-355a-456d-bafc-49985114d7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:08.113380 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1052/2127] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:08.122828 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6dca5-c747-4852-b107-d0cdf6828446 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.128220 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6dca5-c747-4852-b107-d0cdf6828446 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.128220 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6dca5-c747-4852-b107-d0cdf6828446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.143315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-cad6dca5-c747-4852-b107-d0cdf6828446 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:08.144515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-cad6dca5-c747-4852-b107-d0cdf6828446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:08.152330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1053/2128] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:08.156304 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6dca5-c747-4852-b107-d0cdf6828446 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:08.165578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6dca5-c747-4852-b107-d0cdf6828446 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:08.170190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01057e-ecd1-4af7-bef7-40d5830af382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:08.170831 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01057e-ecd1-4af7-bef7-40d5830af382 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.171073 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01057e-ecd1-4af7-bef7-40d5830af382 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.176190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01057e-ecd1-4af7-bef7-40d5830af382 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.176190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01057e-ecd1-4af7-bef7-40d5830af382 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:08.176190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01057e-ecd1-4af7-bef7-40d5830af382 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:08.176190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01057e-ecd1-4af7-bef7-40d5830af382 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:08.183037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1076/2129] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:07 2022] POST /identity/v3/projects => generated 387 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:08.183997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d9ac9-a80b-4798-8b45-b150a3dc89e2 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:08.185621 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d81d9ac9-a80b-4798-8b45-b150a3dc89e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:08.189997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1054/2130] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:58:08.191006 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01057e-ecd1-4af7-bef7-40d5830af382 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:08.200748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01057e-ecd1-4af7-bef7-40d5830af382 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:08.258254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d9ac9-a80b-4798-8b45-b150a3dc89e2 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:08.258707 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86639be1-dec4-4358-9cbd-2ecccde79467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:08.259347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86639be1-dec4-4358-9cbd-2ecccde79467 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.259628 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86639be1-dec4-4358-9cbd-2ecccde79467 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.259898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86639be1-dec4-4358-9cbd-2ecccde79467 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.260528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86639be1-dec4-4358-9cbd-2ecccde79467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:08.262903 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86639be1-dec4-4358-9cbd-2ecccde79467 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:08.263351 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1e4d5-f08f-4f03-ad2c-57de01119b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:08.263946 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1e4d5-f08f-4f03-ad2c-57de01119b7b admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.264223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1e4d5-f08f-4f03-ad2c-57de01119b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.264496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1e4d5-f08f-4f03-ad2c-57de01119b7b admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.265127 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1e4d5-f08f-4f03-ad2c-57de01119b7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:08.267445 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1e4d5-f08f-4f03-ad2c-57de01119b7b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:08.396792 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1055/2131] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:08 2022] POST /identity/v3/projects => generated 391 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:08.401050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86639be1-dec4-4358-9cbd-2ecccde79467 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:08.421564 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:08.444078 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1077/2132] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:08 2022] POST /identity/v3/users => generated 300 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:08.445437 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1e4d5-f08f-4f03-ad2c-57de01119b7b admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:08.455383 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1e4d5-f08f-4f03-ad2c-57de01119b7b admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:08.460851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf64f05-624e-4851-b33a-88d37c28265f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:08.461492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf64f05-624e-4851-b33a-88d37c28265f admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.461775 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf64f05-624e-4851-b33a-88d37c28265f admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.462027 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf64f05-624e-4851-b33a-88d37c28265f admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.462687 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf64f05-624e-4851-b33a-88d37c28265f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:08.464411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf64f05-624e-4851-b33a-88d37c28265f admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:08.465870 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86639be1-dec4-4358-9cbd-2ecccde79467 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:08.470865 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8187a9d-95a6-44ae-9738-608ff151865f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:08.471510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8187a9d-95a6-44ae-9738-608ff151865f admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.471799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8187a9d-95a6-44ae-9738-608ff151865f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.472069 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8187a9d-95a6-44ae-9738-608ff151865f admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.472839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8187a9d-95a6-44ae-9738-608ff151865f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:08.474748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8187a9d-95a6-44ae-9738-608ff151865f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:08.478491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1078/2133] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:08.482739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf64f05-624e-4851-b33a-88d37c28265f admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:08.493908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf64f05-624e-4851-b33a-88d37c28265f admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:08.498736 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471a5ed-a216-4547-950e-df51c1b971d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:08.499513 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471a5ed-a216-4547-950e-df51c1b971d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.499808 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471a5ed-a216-4547-950e-df51c1b971d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.500083 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471a5ed-a216-4547-950e-df51c1b971d7 admin admin] PATH_INFO: `/v3/projects/1c895d7bedd34896b55703c9e17c56f4/users/b9d32cd71e0c4feb874bd2256b7c7116/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.515477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471a5ed-a216-4547-950e-df51c1b971d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c895d7bedd34896b55703c9e17c56f4, user_id=b9d32cd71e0c4feb874bd2256b7c7116, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:08.518127 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471a5ed-a216-4547-950e-df51c1b971d7 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:08.546449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1056/2134] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:08 2022] POST /identity/v3/users => generated 296 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:08.550708 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8187a9d-95a6-44ae-9738-608ff151865f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:08.561341 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8187a9d-95a6-44ae-9738-608ff151865f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:08.566018 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66cb03-8cbe-40c7-bc7c-e98ad9de3892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:08.566682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66cb03-8cbe-40c7-bc7c-e98ad9de3892 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.566965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66cb03-8cbe-40c7-bc7c-e98ad9de3892 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.567268 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66cb03-8cbe-40c7-bc7c-e98ad9de3892 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.567917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66cb03-8cbe-40c7-bc7c-e98ad9de3892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:08.569626 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66cb03-8cbe-40c7-bc7c-e98ad9de3892 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:08.582694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1057/2135] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:08.583795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66cb03-8cbe-40c7-bc7c-e98ad9de3892 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:08.592630 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1079/2136] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:08 2022] PUT /identity/v3/projects/1c895d7bedd34896b55703c9e17c56f4/users/b9d32cd71e0c4feb874bd2256b7c7116/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:08.597106 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66cb03-8cbe-40c7-bc7c-e98ad9de3892 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:08.599764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471a5ed-a216-4547-950e-df51c1b971d7 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:08.629503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:08.662341 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffb252-0384-4da5-be68-8ebf8022a682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:08.662931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffb252-0384-4da5-be68-8ebf8022a682 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.663319 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffb252-0384-4da5-be68-8ebf8022a682 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.664081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffb252-0384-4da5-be68-8ebf8022a682 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.664729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffb252-0384-4da5-be68-8ebf8022a682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:08.665416 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471a5ed-a216-4547-950e-df51c1b971d7 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:08.666461 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffb252-0384-4da5-be68-8ebf8022a682 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:08.670088 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090815d-bec2-4b54-913f-c80588bc7ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:08.670683 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090815d-bec2-4b54-913f-c80588bc7ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.670956 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090815d-bec2-4b54-913f-c80588bc7ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.671258 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090815d-bec2-4b54-913f-c80588bc7ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.671897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090815d-bec2-4b54-913f-c80588bc7ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:08.673460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090815d-bec2-4b54-913f-c80588bc7ca6 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:08.677657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1058/2137] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:08.681972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffb252-0384-4da5-be68-8ebf8022a682 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:08.686500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1080/2138] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:08.687704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090815d-bec2-4b54-913f-c80588bc7ca6 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:08.691640 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffb252-0384-4da5-be68-8ebf8022a682 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:08.697278 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090815d-bec2-4b54-913f-c80588bc7ca6 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:08.697607 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647d05d-e4da-4cfc-9bda-acaa11b38b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:08.698293 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647d05d-e4da-4cfc-9bda-acaa11b38b58 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.698571 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647d05d-e4da-4cfc-9bda-acaa11b38b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.698870 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647d05d-e4da-4cfc-9bda-acaa11b38b58 admin admin] PATH_INFO: `/v3/projects/af72caacf8b34bb29ad2c23dd9ab1fb1/users/662fefd6f27f48aeada7cfc30fcdc13b/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.702891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfcf27-eeb5-4c94-8408-2056a2232aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:08.703549 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfcf27-eeb5-4c94-8408-2056a2232aa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.703938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfcf27-eeb5-4c94-8408-2056a2232aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.704212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfcf27-eeb5-4c94-8408-2056a2232aa9 admin admin] PATH_INFO: `/v3/projects/1c895d7bedd34896b55703c9e17c56f4/users/b9d32cd71e0c4feb874bd2256b7c7116/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.706194 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cfcf27-eeb5-4c94-8408-2056a2232aa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c895d7bedd34896b55703c9e17c56f4, user_id=b9d32cd71e0c4feb874bd2256b7c7116, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:08.708950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cfcf27-eeb5-4c94-8408-2056a2232aa9 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:08.714692 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2647d05d-e4da-4cfc-9bda-acaa11b38b58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af72caacf8b34bb29ad2c23dd9ab1fb1, user_id=662fefd6f27f48aeada7cfc30fcdc13b, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:08.717408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2647d05d-e4da-4cfc-9bda-acaa11b38b58 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:08.720079 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1081/2139] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:08 2022] PUT /identity/v3/projects/1c895d7bedd34896b55703c9e17c56f4/users/b9d32cd71e0c4feb874bd2256b7c7116/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:08.731978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b5771-7b69-40b2-99c9-e8fe47329651 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.732263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b5771-7b69-40b2-99c9-e8fe47329651 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.732536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b5771-7b69-40b2-99c9-e8fe47329651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.760453 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-203b5771-7b69-40b2-99c9-e8fe47329651 None None] MFA Rules not processed for user `b9d32cd71e0c4feb874bd2256b7c7116`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:08.761684 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-203b5771-7b69-40b2-99c9-e8fe47329651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:08.797291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1059/2140] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:08 2022] PUT /identity/v3/projects/af72caacf8b34bb29ad2c23dd9ab1fb1/users/662fefd6f27f48aeada7cfc30fcdc13b/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:08.808177 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a337261-6658-48d7-a0df-2abb2786ef92 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.808554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a337261-6658-48d7-a0df-2abb2786ef92 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.808900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a337261-6658-48d7-a0df-2abb2786ef92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.829648 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1082/2141] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:08.838650 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1a337261-6658-48d7-a0df-2abb2786ef92 None None] MFA Rules not processed for user `662fefd6f27f48aeada7cfc30fcdc13b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:08.839895 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1a337261-6658-48d7-a0df-2abb2786ef92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:08.840847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b5771-7b69-40b2-99c9-e8fe47329651 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:08.842205 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-203b5771-7b69-40b2-99c9-e8fe47329651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:08.898568 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b5771-7b69-40b2-99c9-e8fe47329651 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:08.902648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:08.906280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1060/2142] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:08.910629 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a337261-6658-48d7-a0df-2abb2786ef92 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:08.943621 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:08.948806 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7e191-0a06-44d5-b327-c9f53ada7722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:08.949406 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7e191-0a06-44d5-b327-c9f53ada7722 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.949770 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7e191-0a06-44d5-b327-c9f53ada7722 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.950038 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f7e191-0a06-44d5-b327-c9f53ada7722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.970862 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a337261-6658-48d7-a0df-2abb2786ef92 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:08.976153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63736558-bc00-4ffc-a6e9-1796ad32d8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:08.976936 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63736558-bc00-4ffc-a6e9-1796ad32d8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:08.977218 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63736558-bc00-4ffc-a6e9-1796ad32d8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:08.977610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63736558-bc00-4ffc-a6e9-1796ad32d8a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:08.978286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63736558-bc00-4ffc-a6e9-1796ad32d8a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:08.980279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63736558-bc00-4ffc-a6e9-1796ad32d8a8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:09.014755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f7e191-0a06-44d5-b327-c9f53ada7722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:09.016216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f7e191-0a06-44d5-b327-c9f53ada7722 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:09.025731 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1083/2143] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:58:09.072900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1061/2144] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:08 2022] POST /identity/v3/projects => generated 393 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:09.077012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7e191-0a06-44d5-b327-c9f53ada7722 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:09.087781 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f7e191-0a06-44d5-b327-c9f53ada7722 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:09.154557 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd194ed9-87c3-49cf-91da-43eddedd565f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:09.155186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd194ed9-87c3-49cf-91da-43eddedd565f admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:09.155530 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd194ed9-87c3-49cf-91da-43eddedd565f admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:09.155820 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd194ed9-87c3-49cf-91da-43eddedd565f admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:09.156470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd194ed9-87c3-49cf-91da-43eddedd565f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:09.158392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd194ed9-87c3-49cf-91da-43eddedd565f admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:09.274027 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1084/2145] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:09 2022] POST /identity/v3/users => generated 303 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:09.276891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63736558-bc00-4ffc-a6e9-1796ad32d8a8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:09.286926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63736558-bc00-4ffc-a6e9-1796ad32d8a8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:09.292112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85818bcc-3b0f-4352-97bf-1f9fa8a0f5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:09.292748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85818bcc-3b0f-4352-97bf-1f9fa8a0f5a0 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:09.293027 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85818bcc-3b0f-4352-97bf-1f9fa8a0f5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:09.293306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85818bcc-3b0f-4352-97bf-1f9fa8a0f5a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:09.293957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85818bcc-3b0f-4352-97bf-1f9fa8a0f5a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:09.295595 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85818bcc-3b0f-4352-97bf-1f9fa8a0f5a0 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:09.305374 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1062/2146] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:09.309371 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd194ed9-87c3-49cf-91da-43eddedd565f admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:09.319019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd194ed9-87c3-49cf-91da-43eddedd565f admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:09.323601 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3442af8-2918-47c6-8191-665119fa0ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:09.324228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3442af8-2918-47c6-8191-665119fa0ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:09.324503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3442af8-2918-47c6-8191-665119fa0ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:09.324772 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3442af8-2918-47c6-8191-665119fa0ecb admin admin] PATH_INFO: `/v3/projects/39037882f1804d6f9ad1cb0954b2ee58/users/e6b5afa2aad2437da4e8ce898e35ef9e/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:09.340192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3442af8-2918-47c6-8191-665119fa0ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=39037882f1804d6f9ad1cb0954b2ee58, user_id=e6b5afa2aad2437da4e8ce898e35ef9e, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:09.342811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3442af8-2918-47c6-8191-665119fa0ecb admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:09.437796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1085/2147] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:09 2022] PUT /identity/v3/projects/39037882f1804d6f9ad1cb0954b2ee58/users/e6b5afa2aad2437da4e8ce898e35ef9e/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:09.442012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85818bcc-3b0f-4352-97bf-1f9fa8a0f5a0 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:09.452359 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85818bcc-3b0f-4352-97bf-1f9fa8a0f5a0 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:09.508053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bacb79-096a-451f-a21f-541d57d8cb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:09.508681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bacb79-096a-451f-a21f-541d57d8cb6c admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:09.508964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bacb79-096a-451f-a21f-541d57d8cb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:09.509239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bacb79-096a-451f-a21f-541d57d8cb6c admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:09.509906 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bacb79-096a-451f-a21f-541d57d8cb6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:09.511556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bacb79-096a-451f-a21f-541d57d8cb6c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:09.521182 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1063/2148] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:09.526353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3442af8-2918-47c6-8191-665119fa0ecb admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:09.536326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3442af8-2918-47c6-8191-665119fa0ecb admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:09.541671 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ea448-4e16-47f8-85eb-ff41e5917a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:09.542306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ea448-4e16-47f8-85eb-ff41e5917a12 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:09.542583 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ea448-4e16-47f8-85eb-ff41e5917a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:09.542854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ea448-4e16-47f8-85eb-ff41e5917a12 admin admin] PATH_INFO: `/v3/projects/39037882f1804d6f9ad1cb0954b2ee58/users/e6b5afa2aad2437da4e8ce898e35ef9e/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:09.544995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ea448-4e16-47f8-85eb-ff41e5917a12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39037882f1804d6f9ad1cb0954b2ee58, user_id=e6b5afa2aad2437da4e8ce898e35ef9e, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:09.547804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ea448-4e16-47f8-85eb-ff41e5917a12 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:09.558697 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1086/2149] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:09 2022] PUT /identity/v3/projects/39037882f1804d6f9ad1cb0954b2ee58/users/e6b5afa2aad2437da4e8ce898e35ef9e/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:09.572257 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277f5da-d947-41a9-891a-c6bf14c426a4 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:09.572257 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277f5da-d947-41a9-891a-c6bf14c426a4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:09.572257 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277f5da-d947-41a9-891a-c6bf14c426a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:09.597939 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c277f5da-d947-41a9-891a-c6bf14c426a4 None None] MFA Rules not processed for user `e6b5afa2aad2437da4e8ce898e35ef9e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:09.599197 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c277f5da-d947-41a9-891a-c6bf14c426a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:09.661096 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:09.670302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1064/2150] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:09 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:09.681596 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ea448-4e16-47f8-85eb-ff41e5917a12 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:09.740580 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ea448-4e16-47f8-85eb-ff41e5917a12 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:09.744300 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78b811-3c86-49e6-a3e2-9bcd9f108fd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:09.744900 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78b811-3c86-49e6-a3e2-9bcd9f108fd5 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:09.745178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78b811-3c86-49e6-a3e2-9bcd9f108fd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:09.745447 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78b811-3c86-49e6-a3e2-9bcd9f108fd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:09.751578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:09.755260 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78b811-3c86-49e6-a3e2-9bcd9f108fd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:09.756691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78b811-3c86-49e6-a3e2-9bcd9f108fd5 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:09.766872 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1087/2151] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:09 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:09.815373 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277f5da-d947-41a9-891a-c6bf14c426a4 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:09.825541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277f5da-d947-41a9-891a-c6bf14c426a4 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:09.829147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf59343-7c20-4bbd-b51c-89bd2cadcbfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:09.829738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf59343-7c20-4bbd-b51c-89bd2cadcbfc service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:09.830052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf59343-7c20-4bbd-b51c-89bd2cadcbfc service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:09.830323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf59343-7c20-4bbd-b51c-89bd2cadcbfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:09.887503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf59343-7c20-4bbd-b51c-89bd2cadcbfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:09.888925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf59343-7c20-4bbd-b51c-89bd2cadcbfc service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:09.898305 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1065/2152] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:10.184133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78b811-3c86-49e6-a3e2-9bcd9f108fd5 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:10.185582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-bf78b811-3c86-49e6-a3e2-9bcd9f108fd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:10.196717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78b811-3c86-49e6-a3e2-9bcd9f108fd5 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:10.200425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cfb6c-de63-4e7b-bcfd-4e856c3bfaf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:10.201009 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cfb6c-de63-4e7b-bcfd-4e856c3bfaf8 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:10.201281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cfb6c-de63-4e7b-bcfd-4e856c3bfaf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:10.201545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cfb6c-de63-4e7b-bcfd-4e856c3bfaf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:10.203274 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ee0cfb6c-de63-4e7b-bcfd-4e856c3bfaf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:10.262947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0cfb6c-de63-4e7b-bcfd-4e856c3bfaf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:10.264806 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0cfb6c-de63-4e7b-bcfd-4e856c3bfaf8 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:10.277267 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1088/2153] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:16.886326 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf59343-7c20-4bbd-b51c-89bd2cadcbfc service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:16.956306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf59343-7c20-4bbd-b51c-89bd2cadcbfc service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:16.961449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb1e42-4727-4505-b778-8f9fb5b1f62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:16.962070 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb1e42-4727-4505-b778-8f9fb5b1f62a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:16.962347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb1e42-4727-4505-b778-8f9fb5b1f62a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:16.962617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb1e42-4727-4505-b778-8f9fb5b1f62a admin admin] PATH_INFO: `/v3/users/662fefd6f27f48aeada7cfc30fcdc13b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:16.963748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb1e42-4727-4505-b778-8f9fb5b1f62a admin admin] RBAC: Authorizing `identity:delete_user(user_id=662fefd6f27f48aeada7cfc30fcdc13b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:16.965743 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb1e42-4727-4505-b778-8f9fb5b1f62a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:17.174907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8eeb1e42-4727-4505-b778-8f9fb5b1f62a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '662fefd6f27f48aeada7cfc30fcdc13b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:17.181569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8eeb1e42-4727-4505-b778-8f9fb5b1f62a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '662fefd6f27f48aeada7cfc30fcdc13b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:17.182226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8eeb1e42-4727-4505-b778-8f9fb5b1f62a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '662fefd6f27f48aeada7cfc30fcdc13b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:17.236303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8eeb1e42-4727-4505-b778-8f9fb5b1f62a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '662fefd6f27f48aeada7cfc30fcdc13b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:17.249143 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8eeb1e42-4727-4505-b778-8f9fb5b1f62a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '662fefd6f27f48aeada7cfc30fcdc13b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:17.255299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1066/2154] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:16 2022] DELETE /identity/v3/users/662fefd6f27f48aeada7cfc30fcdc13b => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:17.260281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60ce43-b8af-4db5-9530-c6f1f652cc80 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:17.260741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60ce43-b8af-4db5-9530-c6f1f652cc80 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:17.260832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60ce43-b8af-4db5-9530-c6f1f652cc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:17.281074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-ca60ce43-b8af-4db5-9530-c6f1f652cc80 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:17.282273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ca60ce43-b8af-4db5-9530-c6f1f652cc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:17.341596 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1089/2155] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:17.351531 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb1e42-4727-4505-b778-8f9fb5b1f62a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:17.353045 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8eeb1e42-4727-4505-b778-8f9fb5b1f62a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:17.368741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:17.415055 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb1e42-4727-4505-b778-8f9fb5b1f62a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:17.420637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522723b-53f0-4ca0-a087-103534c7117c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:17.420637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522723b-53f0-4ca0-a087-103534c7117c service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:17.420637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522723b-53f0-4ca0-a087-103534c7117c service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:17.420637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522723b-53f0-4ca0-a087-103534c7117c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:17.429740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522723b-53f0-4ca0-a087-103534c7117c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:17.432608 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522723b-53f0-4ca0-a087-103534c7117c service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:17.442567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1067/2156] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:58:17.731199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60ce43-b8af-4db5-9530-c6f1f652cc80 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:17.732645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ca60ce43-b8af-4db5-9530-c6f1f652cc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:17.796199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60ce43-b8af-4db5-9530-c6f1f652cc80 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:17.800510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e120120-e69c-49bb-afd6-4dabe087f643 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:17.800665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e120120-e69c-49bb-afd6-4dabe087f643 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:17.800898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e120120-e69c-49bb-afd6-4dabe087f643 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:17.801169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e120120-e69c-49bb-afd6-4dabe087f643 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:17.802898 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1e120120-e69c-49bb-afd6-4dabe087f643 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:17.844437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522723b-53f0-4ca0-a087-103534c7117c service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:17.845864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8522723b-53f0-4ca0-a087-103534c7117c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:17.860795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522723b-53f0-4ca0-a087-103534c7117c service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:17.865906 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:17.866616 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:17.866751 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:17.866901 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:17.867919 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06 admin admin] PATH_INFO: `/v3/projects/af72caacf8b34bb29ad2c23dd9ab1fb1` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:17.869197 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af72caacf8b34bb29ad2c23dd9ab1fb1)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:17.872139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:17.888847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e120120-e69c-49bb-afd6-4dabe087f643 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:17.890645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e120120-e69c-49bb-afd6-4dabe087f643 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:17.901839 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1090/2157] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:18.005046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af72caacf8b34bb29ad2c23dd9ab1fb1', 'request_id': 'req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:18.012011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af72caacf8b34bb29ad2c23dd9ab1fb1', 'request_id': 'req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:18.017642 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af72caacf8b34bb29ad2c23dd9ab1fb1', 'request_id': 'req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:18.071000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06 admin admin] The token cache is being invalidate because project af72caacf8b34bb29ad2c23dd9ab1fb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:18.071560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:18.072420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:18.074001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1068/2158] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:17 2022] DELETE /identity/v3/projects/af72caacf8b34bb29ad2c23dd9ab1fb1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:18.078170 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e120120-e69c-49bb-afd6-4dabe087f643 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:18.090118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e120120-e69c-49bb-afd6-4dabe087f643 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:18.146364 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44c2a4-7080-4ddf-b9c6-cb9ecf985947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:18.146984 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44c2a4-7080-4ddf-b9c6-cb9ecf985947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:18.147293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44c2a4-7080-4ddf-b9c6-cb9ecf985947 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:18.147565 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44c2a4-7080-4ddf-b9c6-cb9ecf985947 admin admin] PATH_INFO: `/v3/users/e6b5afa2aad2437da4e8ce898e35ef9e` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:18.149340 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44c2a4-7080-4ddf-b9c6-cb9ecf985947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6b5afa2aad2437da4e8ce898e35ef9e)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:18.151306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44c2a4-7080-4ddf-b9c6-cb9ecf985947 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:18.369173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7b44c2a4-7080-4ddf-b9c6-cb9ecf985947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6b5afa2aad2437da4e8ce898e35ef9e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:18.376646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7b44c2a4-7080-4ddf-b9c6-cb9ecf985947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6b5afa2aad2437da4e8ce898e35ef9e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:18.391790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7b44c2a4-7080-4ddf-b9c6-cb9ecf985947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6b5afa2aad2437da4e8ce898e35ef9e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:18.392597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7b44c2a4-7080-4ddf-b9c6-cb9ecf985947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6b5afa2aad2437da4e8ce898e35ef9e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:18.393277 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7b44c2a4-7080-4ddf-b9c6-cb9ecf985947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6b5afa2aad2437da4e8ce898e35ef9e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:18.484492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1091/2159] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:18 2022] DELETE /identity/v3/users/e6b5afa2aad2437da4e8ce898e35ef9e => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:19.013939 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:19.015505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:19.034814 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa65ac-bc34-4437-9fb4-ba9ddcc2eb06 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:19.092748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41008075-ff0e-4653-8db5-0a36037ebe89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:19.093504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41008075-ff0e-4653-8db5-0a36037ebe89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:19.093782 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41008075-ff0e-4653-8db5-0a36037ebe89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:19.094053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41008075-ff0e-4653-8db5-0a36037ebe89 admin admin] PATH_INFO: `/v3/projects/39037882f1804d6f9ad1cb0954b2ee58` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:19.095325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41008075-ff0e-4653-8db5-0a36037ebe89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39037882f1804d6f9ad1cb0954b2ee58)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:19.097257 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41008075-ff0e-4653-8db5-0a36037ebe89 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:19.201125 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:19.206990 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-41008075-ff0e-4653-8db5-0a36037ebe89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39037882f1804d6f9ad1cb0954b2ee58', 'request_id': 'req-41008075-ff0e-4653-8db5-0a36037ebe89'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:19.213711 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-41008075-ff0e-4653-8db5-0a36037ebe89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39037882f1804d6f9ad1cb0954b2ee58', 'request_id': 'req-41008075-ff0e-4653-8db5-0a36037ebe89'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:19.219299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-41008075-ff0e-4653-8db5-0a36037ebe89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39037882f1804d6f9ad1cb0954b2ee58', 'request_id': 'req-41008075-ff0e-4653-8db5-0a36037ebe89'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:19.298603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-41008075-ff0e-4653-8db5-0a36037ebe89 admin admin] The token cache is being invalidate because project 39037882f1804d6f9ad1cb0954b2ee58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:19.298885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-41008075-ff0e-4653-8db5-0a36037ebe89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:19.299521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-41008075-ff0e-4653-8db5-0a36037ebe89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:19.301148 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1069/2160] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:19 2022] DELETE /identity/v3/projects/39037882f1804d6f9ad1cb0954b2ee58 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:19.306098 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1da26f-cd1a-4c71-967c-c0e7ffe21272 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:19.306381 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1da26f-cd1a-4c71-967c-c0e7ffe21272 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:19.306648 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1da26f-cd1a-4c71-967c-c0e7ffe21272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:19.332527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-6c1da26f-cd1a-4c71-967c-c0e7ffe21272 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:19.333779 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6c1da26f-cd1a-4c71-967c-c0e7ffe21272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:19.412369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1092/2161] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:19.420575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ab6c2-4ceb-4e92-aa37-84ad62b1fb59 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:19.420917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ab6c2-4ceb-4e92-aa37-84ad62b1fb59 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:19.421198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ab6c2-4ceb-4e92-aa37-84ad62b1fb59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:19.441953 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-7c7ab6c2-4ceb-4e92-aa37-84ad62b1fb59 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:19.443428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7c7ab6c2-4ceb-4e92-aa37-84ad62b1fb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:19.454704 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1070/2162] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:19.457980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1da26f-cd1a-4c71-967c-c0e7ffe21272 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:19.469125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1da26f-cd1a-4c71-967c-c0e7ffe21272 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:19.474203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea7e4-7b45-49f0-a23a-96795aebedea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:19.474835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea7e4-7b45-49f0-a23a-96795aebedea admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:19.475133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea7e4-7b45-49f0-a23a-96795aebedea admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:19.475448 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbea7e4-7b45-49f0-a23a-96795aebedea admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:19.476108 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbea7e4-7b45-49f0-a23a-96795aebedea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:19.476928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbea7e4-7b45-49f0-a23a-96795aebedea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:19.478696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbea7e4-7b45-49f0-a23a-96795aebedea admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:19.480869 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:19.496846 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1093/2163] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:58:19.500649 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ab6c2-4ceb-4e92-aa37-84ad62b1fb59 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:19.511118 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ab6c2-4ceb-4e92-aa37-84ad62b1fb59 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:19.516543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231afaa-e3f7-4a50-b5d0-566620dcd317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:19.517658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231afaa-e3f7-4a50-b5d0-566620dcd317 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:19.518907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231afaa-e3f7-4a50-b5d0-566620dcd317 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:19.519404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231afaa-e3f7-4a50-b5d0-566620dcd317 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:19.520577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3231afaa-e3f7-4a50-b5d0-566620dcd317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:19.528285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3231afaa-e3f7-4a50-b5d0-566620dcd317 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:19.638053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1071/2164] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:19 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:19.639166 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea7e4-7b45-49f0-a23a-96795aebedea admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:19.649461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbea7e4-7b45-49f0-a23a-96795aebedea admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:19.705646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5019873-06cc-480d-a572-29338f7cf887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:19.706255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5019873-06cc-480d-a572-29338f7cf887 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:19.706529 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5019873-06cc-480d-a572-29338f7cf887 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:19.706801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5019873-06cc-480d-a572-29338f7cf887 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:19.707471 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5019873-06cc-480d-a572-29338f7cf887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:19.710377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5019873-06cc-480d-a572-29338f7cf887 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:19.818581 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1094/2165] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:19 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:19.830200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231afaa-e3f7-4a50-b5d0-566620dcd317 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:19.840892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231afaa-e3f7-4a50-b5d0-566620dcd317 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:19.845540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4c76d-643e-45a5-804b-564560af192a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:19.846145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4c76d-643e-45a5-804b-564560af192a admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:19.846427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4c76d-643e-45a5-804b-564560af192a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:19.846701 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4c76d-643e-45a5-804b-564560af192a admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:19.847420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4c76d-643e-45a5-804b-564560af192a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:19.849055 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4c76d-643e-45a5-804b-564560af192a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:19.861381 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1072/2166] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:19.865439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5019873-06cc-480d-a572-29338f7cf887 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:19.875575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5019873-06cc-480d-a572-29338f7cf887 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:19.880130 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78981f0-dd65-4e35-bd8b-0851df1c4bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:19.880725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78981f0-dd65-4e35-bd8b-0851df1c4bcb admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:19.880996 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78981f0-dd65-4e35-bd8b-0851df1c4bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:19.881264 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78981f0-dd65-4e35-bd8b-0851df1c4bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:19.881899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78981f0-dd65-4e35-bd8b-0851df1c4bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:19.883556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78981f0-dd65-4e35-bd8b-0851df1c4bcb admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:19.893985 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1095/2167] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:19.898676 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4c76d-643e-45a5-804b-564560af192a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:19.908504 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4c76d-643e-45a5-804b-564560af192a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:19.913163 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dac7b-505b-44db-9c57-ac0f30c51a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:19.914493 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dac7b-505b-44db-9c57-ac0f30c51a48 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:19.914792 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dac7b-505b-44db-9c57-ac0f30c51a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:19.915066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1dac7b-505b-44db-9c57-ac0f30c51a48 admin admin] PATH_INFO: `/v3/projects/800bd58c1c97462fa2219507150adce3/users/b3a255724c404cfc990343a147093cc5/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:19.929020 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1dac7b-505b-44db-9c57-ac0f30c51a48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=800bd58c1c97462fa2219507150adce3, user_id=b3a255724c404cfc990343a147093cc5, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:19.931685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1dac7b-505b-44db-9c57-ac0f30c51a48 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:20.030718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1073/2168] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:19 2022] PUT /identity/v3/projects/800bd58c1c97462fa2219507150adce3/users/b3a255724c404cfc990343a147093cc5/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:20.041930 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba17eca-5767-45f5-b09f-7acc9ef9cc1a None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:20.042217 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba17eca-5767-45f5-b09f-7acc9ef9cc1a None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:20.042488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba17eca-5767-45f5-b09f-7acc9ef9cc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:20.074456 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-4ba17eca-5767-45f5-b09f-7acc9ef9cc1a None None] MFA Rules not processed for user `b3a255724c404cfc990343a147093cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:20.074456 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-4ba17eca-5767-45f5-b09f-7acc9ef9cc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:20.137095 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1096/2169] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:20.147928 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dac7b-505b-44db-9c57-ac0f30c51a48 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:20.150285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1c1dac7b-505b-44db-9c57-ac0f30c51a48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:20.206759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1dac7b-505b-44db-9c57-ac0f30c51a48 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:20.210631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ccd62-c202-4e31-b4d6-ede3eabc76a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:20.211872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ccd62-c202-4e31-b4d6-ede3eabc76a3 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:20.212140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ccd62-c202-4e31-b4d6-ede3eabc76a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:20.212413 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ccd62-c202-4e31-b4d6-ede3eabc76a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:20.221528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ccd62-c202-4e31-b4d6-ede3eabc76a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:20.222958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ccd62-c202-4e31-b4d6-ede3eabc76a3 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:20.236902 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1074/2170] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:20.252196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba17eca-5767-45f5-b09f-7acc9ef9cc1a None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:20.320637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba17eca-5767-45f5-b09f-7acc9ef9cc1a None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:20.325400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf201d-09c5-4a7d-8dad-31301d1356cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:20.325956 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf201d-09c5-4a7d-8dad-31301d1356cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:20.326231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf201d-09c5-4a7d-8dad-31301d1356cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:20.326495 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf201d-09c5-4a7d-8dad-31301d1356cc admin admin] PATH_INFO: `/v3/users/b3a255724c404cfc990343a147093cc5` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:20.327680 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf201d-09c5-4a7d-8dad-31301d1356cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a255724c404cfc990343a147093cc5)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:20.329622 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf201d-09c5-4a7d-8dad-31301d1356cc admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:20.331978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:20.667369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ffaf201d-09c5-4a7d-8dad-31301d1356cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a255724c404cfc990343a147093cc5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:20.674748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ffaf201d-09c5-4a7d-8dad-31301d1356cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a255724c404cfc990343a147093cc5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:20.687756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ffaf201d-09c5-4a7d-8dad-31301d1356cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a255724c404cfc990343a147093cc5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:20.688565 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ffaf201d-09c5-4a7d-8dad-31301d1356cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a255724c404cfc990343a147093cc5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:20.689245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ffaf201d-09c5-4a7d-8dad-31301d1356cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a255724c404cfc990343a147093cc5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:20.773478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1097/2171] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:20 2022] DELETE /identity/v3/users/b3a255724c404cfc990343a147093cc5 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:20.778218 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b12da-5cca-450e-8496-0bb5cea405f4 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:20.778535 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b12da-5cca-450e-8496-0bb5cea405f4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:20.778810 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b12da-5cca-450e-8496-0bb5cea405f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:20.797951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-273b12da-5cca-450e-8496-0bb5cea405f4 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:20.803441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-273b12da-5cca-450e-8496-0bb5cea405f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:20.860414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1075/2172] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:20.879025 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf201d-09c5-4a7d-8dad-31301d1356cc admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:20.881024 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ffaf201d-09c5-4a7d-8dad-31301d1356cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:20.950566 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf201d-09c5-4a7d-8dad-31301d1356cc admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:20.954116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72acb3-ea88-497d-bb37-1906155e6971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:20.954692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72acb3-ea88-497d-bb37-1906155e6971 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:20.954968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72acb3-ea88-497d-bb37-1906155e6971 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:20.955265 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72acb3-ea88-497d-bb37-1906155e6971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:20.964215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72acb3-ea88-497d-bb37-1906155e6971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:20.965620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72acb3-ea88-497d-bb37-1906155e6971 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:20.975652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1098/2173] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:21.500769 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b12da-5cca-450e-8496-0bb5cea405f4 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:21.505210 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-273b12da-5cca-450e-8496-0bb5cea405f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:21.514357 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b12da-5cca-450e-8496-0bb5cea405f4 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:21.519555 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55c0c0-662b-49cd-87e7-a4c3d15679b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:21.520187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55c0c0-662b-49cd-87e7-a4c3d15679b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:21.520474 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55c0c0-662b-49cd-87e7-a4c3d15679b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:21.520743 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55c0c0-662b-49cd-87e7-a4c3d15679b5 admin admin] PATH_INFO: `/v3/projects/800bd58c1c97462fa2219507150adce3` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:21.521853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55c0c0-662b-49cd-87e7-a4c3d15679b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=800bd58c1c97462fa2219507150adce3)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:21.524389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55c0c0-662b-49cd-87e7-a4c3d15679b5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:21.526881 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:21.651145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7f55c0c0-662b-49cd-87e7-a4c3d15679b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '800bd58c1c97462fa2219507150adce3', 'request_id': 'req-7f55c0c0-662b-49cd-87e7-a4c3d15679b5'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:21.657606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7f55c0c0-662b-49cd-87e7-a4c3d15679b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '800bd58c1c97462fa2219507150adce3', 'request_id': 'req-7f55c0c0-662b-49cd-87e7-a4c3d15679b5'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:21.663606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7f55c0c0-662b-49cd-87e7-a4c3d15679b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '800bd58c1c97462fa2219507150adce3', 'request_id': 'req-7f55c0c0-662b-49cd-87e7-a4c3d15679b5'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:21.731799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7f55c0c0-662b-49cd-87e7-a4c3d15679b5 admin admin] The token cache is being invalidate because project 800bd58c1c97462fa2219507150adce3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:21.732199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7f55c0c0-662b-49cd-87e7-a4c3d15679b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:21.732897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7f55c0c0-662b-49cd-87e7-a4c3d15679b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:21.734768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1076/2174] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:21 2022] DELETE /identity/v3/projects/800bd58c1c97462fa2219507150adce3 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:21.741946 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d9848-17ab-46c4-b204-3f5c0238b487 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:21.741946 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d9848-17ab-46c4-b204-3f5c0238b487 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:21.741946 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d9848-17ab-46c4-b204-3f5c0238b487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:21.760713 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-956d9848-17ab-46c4-b204-3f5c0238b487 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:21.761927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-956d9848-17ab-46c4-b204-3f5c0238b487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:21.822769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1099/2175] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:21.831385 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e531e1a-82ab-4f26-8c32-e3fcca623e59 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:21.831671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e531e1a-82ab-4f26-8c32-e3fcca623e59 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:21.831944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e531e1a-82ab-4f26-8c32-e3fcca623e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:21.852429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-9e531e1a-82ab-4f26-8c32-e3fcca623e59 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:21.856684 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9e531e1a-82ab-4f26-8c32-e3fcca623e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:21.862228 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1077/2176] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:21.866551 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d9848-17ab-46c4-b204-3f5c0238b487 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:21.876666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d9848-17ab-46c4-b204-3f5c0238b487 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:21.881606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404a799-5b91-4450-bbf6-d5456df434aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:21.882252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404a799-5b91-4450-bbf6-d5456df434aa admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:21.883118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404a799-5b91-4450-bbf6-d5456df434aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:21.883381 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404a799-5b91-4450-bbf6-d5456df434aa admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:21.884029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404a799-5b91-4450-bbf6-d5456df434aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:21.884330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404a799-5b91-4450-bbf6-d5456df434aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:21.885918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404a799-5b91-4450-bbf6-d5456df434aa admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:21.897601 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1100/2177] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:58:21.901899 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e531e1a-82ab-4f26-8c32-e3fcca623e59 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:21.911415 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e531e1a-82ab-4f26-8c32-e3fcca623e59 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:21.916780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc990c74-7630-4496-aeb9-0b976665a655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:21.917568 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc990c74-7630-4496-aeb9-0b976665a655 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:21.917865 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc990c74-7630-4496-aeb9-0b976665a655 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:21.918130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc990c74-7630-4496-aeb9-0b976665a655 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:21.918770 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc990c74-7630-4496-aeb9-0b976665a655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:21.920715 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc990c74-7630-4496-aeb9-0b976665a655 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:22.049424 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1078/2178] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:21 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:22.053683 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404a799-5b91-4450-bbf6-d5456df434aa admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:22.064098 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404a799-5b91-4450-bbf6-d5456df434aa admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:22.072364 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:22.120964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4edfb7-52dd-438a-b189-298a67fe057b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:22.121564 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4edfb7-52dd-438a-b189-298a67fe057b admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:22.121844 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4edfb7-52dd-438a-b189-298a67fe057b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:22.122116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4edfb7-52dd-438a-b189-298a67fe057b admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:22.122740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4edfb7-52dd-438a-b189-298a67fe057b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:22.124690 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4edfb7-52dd-438a-b189-298a67fe057b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:22.225386 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1101/2179] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:22 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:22.230422 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc990c74-7630-4496-aeb9-0b976665a655 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:22.230422 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bc990c74-7630-4496-aeb9-0b976665a655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:22.231404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4edfb7-52dd-438a-b189-298a67fe057b admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:22.241183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4edfb7-52dd-438a-b189-298a67fe057b admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:22.241481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc990c74-7630-4496-aeb9-0b976665a655 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:22.246319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fac29-6593-497f-8728-b18f71e71ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:22.251361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd2e53-cee9-4ea0-86ed-a0f7f8155736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:22.251498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fac29-6593-497f-8728-b18f71e71ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:22.251498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fac29-6593-497f-8728-b18f71e71ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:22.251757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd2e53-cee9-4ea0-86ed-a0f7f8155736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:22.251757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd2e53-cee9-4ea0-86ed-a0f7f8155736 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:22.251930 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fac29-6593-497f-8728-b18f71e71ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:22.252021 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd2e53-cee9-4ea0-86ed-a0f7f8155736 admin admin] PATH_INFO: `/v3/users/603e331893634640986a040ecb4dd64d` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:22.252118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fac29-6593-497f-8728-b18f71e71ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:22.252205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd2e53-cee9-4ea0-86ed-a0f7f8155736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=603e331893634640986a040ecb4dd64d)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:22.252314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fac29-6593-497f-8728-b18f71e71ca4 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:22.252405 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd2e53-cee9-4ea0-86ed-a0f7f8155736 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:22.263860 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1102/2180] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:22.267932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fac29-6593-497f-8728-b18f71e71ca4 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:22.278129 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fac29-6593-497f-8728-b18f71e71ca4 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:22.283070 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcafd7-ea27-482e-adfe-ac630c3d7afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:22.283677 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcafd7-ea27-482e-adfe-ac630c3d7afa admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:22.283950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcafd7-ea27-482e-adfe-ac630c3d7afa admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:22.284215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcafd7-ea27-482e-adfe-ac630c3d7afa admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:22.284855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efcafd7-ea27-482e-adfe-ac630c3d7afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:22.286457 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efcafd7-ea27-482e-adfe-ac630c3d7afa admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:22.296208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1103/2181] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:22.300363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcafd7-ea27-482e-adfe-ac630c3d7afa admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:22.310130 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcafd7-ea27-482e-adfe-ac630c3d7afa admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:22.314600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6b866-fb86-40de-8f64-b687920a59de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:22.315230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6b866-fb86-40de-8f64-b687920a59de admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:22.322696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6b866-fb86-40de-8f64-b687920a59de admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:22.322696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6b866-fb86-40de-8f64-b687920a59de admin admin] PATH_INFO: `/v3/projects/25e51d4495ee40bc86874411577cb6b8/users/5087516eb54a40f291c9ad0afbd1896d/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:22.330326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6b866-fb86-40de-8f64-b687920a59de admin admin] RBAC: Authorizing `identity:create_grant(project_id=25e51d4495ee40bc86874411577cb6b8, user_id=5087516eb54a40f291c9ad0afbd1896d, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:22.332952 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6b866-fb86-40de-8f64-b687920a59de admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:22.412099 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1104/2182] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:22 2022] PUT /identity/v3/projects/25e51d4495ee40bc86874411577cb6b8/users/5087516eb54a40f291c9ad0afbd1896d/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:22.422654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b9f62-4a16-441f-93b9-d73aaca18bfe None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:22.422934 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b9f62-4a16-441f-93b9-d73aaca18bfe None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:22.423243 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b9f62-4a16-441f-93b9-d73aaca18bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:22.450963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-61cd2e53-cee9-4ea0-86ed-a0f7f8155736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '603e331893634640986a040ecb4dd64d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:22.451726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-7b9b9f62-4a16-441f-93b9-d73aaca18bfe None None] MFA Rules not processed for user `5087516eb54a40f291c9ad0afbd1896d`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:22.452946 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7b9b9f62-4a16-441f-93b9-d73aaca18bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:22.458388 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-61cd2e53-cee9-4ea0-86ed-a0f7f8155736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '603e331893634640986a040ecb4dd64d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:22.459127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-61cd2e53-cee9-4ea0-86ed-a0f7f8155736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '603e331893634640986a040ecb4dd64d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:22.516088 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1105/2183] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:22.526976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b9f62-4a16-441f-93b9-d73aaca18bfe None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:22.529197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7b9b9f62-4a16-441f-93b9-d73aaca18bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:22.550429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-61cd2e53-cee9-4ea0-86ed-a0f7f8155736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '603e331893634640986a040ecb4dd64d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:22.564359 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-61cd2e53-cee9-4ea0-86ed-a0f7f8155736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '603e331893634640986a040ecb4dd64d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:22.570627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1079/2184] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:22 2022] DELETE /identity/v3/users/603e331893634640986a040ecb4dd64d => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:22.574893 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d27389-804c-4407-adb9-68d273fc94a8 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:22.575260 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d27389-804c-4407-adb9-68d273fc94a8 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:22.575534 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d27389-804c-4407-adb9-68d273fc94a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:22.577009 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:22.589702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b9f62-4a16-441f-93b9-d73aaca18bfe None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:22.593913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-42d27389-804c-4407-adb9-68d273fc94a8 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:22.595146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-42d27389-804c-4407-adb9-68d273fc94a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:22.625885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:22.639041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bcd83-2cf1-4550-9438-b303e8d1e485 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:22.639746 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bcd83-2cf1-4550-9438-b303e8d1e485 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:22.640029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bcd83-2cf1-4550-9438-b303e8d1e485 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:22.640296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bcd83-2cf1-4550-9438-b303e8d1e485 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:22.653480 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1080/2185] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:22.663136 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d27389-804c-4407-adb9-68d273fc94a8 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:22.665019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-42d27389-804c-4407-adb9-68d273fc94a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:22.696366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bcd83-2cf1-4550-9438-b303e8d1e485 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:22.697797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bcd83-2cf1-4550-9438-b303e8d1e485 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:22.706729 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1106/2186] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:58:22.721733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d27389-804c-4407-adb9-68d273fc94a8 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:22.725447 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5d2a0-be10-4749-b64c-66afde219ce3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:22.726048 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5d2a0-be10-4749-b64c-66afde219ce3 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:22.727088 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5d2a0-be10-4749-b64c-66afde219ce3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:22.727457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5d2a0-be10-4749-b64c-66afde219ce3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:22.729318 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bcd83-2cf1-4550-9438-b303e8d1e485 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:22.731046 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-bb8bcd83-2cf1-4550-9438-b303e8d1e485 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:22.736918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5d2a0-be10-4749-b64c-66afde219ce3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:22.738345 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5d2a0-be10-4749-b64c-66afde219ce3 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:22.744574 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bcd83-2cf1-4550-9438-b303e8d1e485 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:22.748122 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1081/2187] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:58:22.752513 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2d732-b8aa-4b1f-8af8-111d9bbc23fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:22.753197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2d732-b8aa-4b1f-8af8-111d9bbc23fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:22.753494 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2d732-b8aa-4b1f-8af8-111d9bbc23fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:22.753761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2d732-b8aa-4b1f-8af8-111d9bbc23fb admin admin] PATH_INFO: `/v3/users/5087516eb54a40f291c9ad0afbd1896d` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:22.754897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2d732-b8aa-4b1f-8af8-111d9bbc23fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5087516eb54a40f291c9ad0afbd1896d)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:22.756885 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2d732-b8aa-4b1f-8af8-111d9bbc23fb admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:23.001606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-bde2d732-b8aa-4b1f-8af8-111d9bbc23fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5087516eb54a40f291c9ad0afbd1896d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.009336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-bde2d732-b8aa-4b1f-8af8-111d9bbc23fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5087516eb54a40f291c9ad0afbd1896d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.021921 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-bde2d732-b8aa-4b1f-8af8-111d9bbc23fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5087516eb54a40f291c9ad0afbd1896d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.022681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-bde2d732-b8aa-4b1f-8af8-111d9bbc23fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5087516eb54a40f291c9ad0afbd1896d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.023884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-bde2d732-b8aa-4b1f-8af8-111d9bbc23fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5087516eb54a40f291c9ad0afbd1896d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.110947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1107/2188] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:22 2022] DELETE /identity/v3/users/5087516eb54a40f291c9ad0afbd1896d => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:23.115965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28918a9-9a43-4e04-80d2-8cd9a88f7979 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:23.116327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28918a9-9a43-4e04-80d2-8cd9a88f7979 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:23.116694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28918a9-9a43-4e04-80d2-8cd9a88f7979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:23.136517 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e28918a9-9a43-4e04-80d2-8cd9a88f7979 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:23.138609 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e28918a9-9a43-4e04-80d2-8cd9a88f7979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:23.195537 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1082/2189] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:23.205405 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2d732-b8aa-4b1f-8af8-111d9bbc23fb admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:23.206811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-bde2d732-b8aa-4b1f-8af8-111d9bbc23fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:23.215986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28918a9-9a43-4e04-80d2-8cd9a88f7979 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:23.217461 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e28918a9-9a43-4e04-80d2-8cd9a88f7979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:23.233604 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28918a9-9a43-4e04-80d2-8cd9a88f7979 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:23.239322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526c30-1821-4861-9675-8f1a3533c681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:23.240060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526c30-1821-4861-9675-8f1a3533c681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:23.242004 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526c30-1821-4861-9675-8f1a3533c681 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:23.242004 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8526c30-1821-4861-9675-8f1a3533c681 admin admin] PATH_INFO: `/v3/projects/87fdafb470e34d01bc58a49d69e3d97c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:23.242418 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8526c30-1821-4861-9675-8f1a3533c681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87fdafb470e34d01bc58a49d69e3d97c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:23.245856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8526c30-1821-4861-9675-8f1a3533c681 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:23.248906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:23.277786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2d732-b8aa-4b1f-8af8-111d9bbc23fb admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:23.281587 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5feea-f045-40ed-b8f8-86c1a6a62abc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:23.282191 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5feea-f045-40ed-b8f8-86c1a6a62abc service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:23.282463 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5feea-f045-40ed-b8f8-86c1a6a62abc service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:23.282730 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5feea-f045-40ed-b8f8-86c1a6a62abc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:23.291743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5feea-f045-40ed-b8f8-86c1a6a62abc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:23.293164 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5feea-f045-40ed-b8f8-86c1a6a62abc service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:23.302839 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1108/2190] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:23.341112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a8526c30-1821-4861-9675-8f1a3533c681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87fdafb470e34d01bc58a49d69e3d97c', 'request_id': 'req-a8526c30-1821-4861-9675-8f1a3533c681'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.347314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a8526c30-1821-4861-9675-8f1a3533c681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87fdafb470e34d01bc58a49d69e3d97c', 'request_id': 'req-a8526c30-1821-4861-9675-8f1a3533c681'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.353511 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a8526c30-1821-4861-9675-8f1a3533c681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87fdafb470e34d01bc58a49d69e3d97c', 'request_id': 'req-a8526c30-1821-4861-9675-8f1a3533c681'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.418548 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a8526c30-1821-4861-9675-8f1a3533c681 admin admin] The token cache is being invalidate because project 87fdafb470e34d01bc58a49d69e3d97c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:23.426613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a8526c30-1821-4861-9675-8f1a3533c681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.426613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a8526c30-1821-4861-9675-8f1a3533c681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.426613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1083/2191] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:23 2022] DELETE /identity/v3/projects/87fdafb470e34d01bc58a49d69e3d97c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:23.426899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5feea-f045-40ed-b8f8-86c1a6a62abc service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:23.434496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5feea-f045-40ed-b8f8-86c1a6a62abc service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:23.490980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706839f-2f23-4dd7-a210-d68aa5e5769f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:23.491631 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706839f-2f23-4dd7-a210-d68aa5e5769f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:23.492411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706839f-2f23-4dd7-a210-d68aa5e5769f admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:23.492683 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706839f-2f23-4dd7-a210-d68aa5e5769f admin admin] PATH_INFO: `/v3/users/00989e8a404546e6a1b1e4db9e06744e` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:23.493822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706839f-2f23-4dd7-a210-d68aa5e5769f admin admin] RBAC: Authorizing `identity:delete_user(user_id=00989e8a404546e6a1b1e4db9e06744e)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:23.495832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706839f-2f23-4dd7-a210-d68aa5e5769f admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:23.707267 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-4706839f-2f23-4dd7-a210-d68aa5e5769f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00989e8a404546e6a1b1e4db9e06744e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.714180 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-4706839f-2f23-4dd7-a210-d68aa5e5769f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00989e8a404546e6a1b1e4db9e06744e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.725983 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-4706839f-2f23-4dd7-a210-d68aa5e5769f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00989e8a404546e6a1b1e4db9e06744e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.726694 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-4706839f-2f23-4dd7-a210-d68aa5e5769f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00989e8a404546e6a1b1e4db9e06744e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.727336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-4706839f-2f23-4dd7-a210-d68aa5e5769f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00989e8a404546e6a1b1e4db9e06744e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.773169 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526c30-1821-4861-9675-8f1a3533c681 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:23.774628 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a8526c30-1821-4861-9675-8f1a3533c681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:23.782605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:23.790737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8526c30-1821-4861-9675-8f1a3533c681 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:23.796146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20758c50-cec9-48d6-a83e-b8974c62c51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:23.796631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20758c50-cec9-48d6-a83e-b8974c62c51f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:23.796925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20758c50-cec9-48d6-a83e-b8974c62c51f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:23.797303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20758c50-cec9-48d6-a83e-b8974c62c51f admin admin] PATH_INFO: `/v3/projects/25e51d4495ee40bc86874411577cb6b8` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:23.798462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20758c50-cec9-48d6-a83e-b8974c62c51f admin admin] RBAC: Authorizing `identity:delete_project(project_id=25e51d4495ee40bc86874411577cb6b8)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:23.800412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20758c50-cec9-48d6-a83e-b8974c62c51f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:23.859233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1109/2192] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:23 2022] DELETE /identity/v3/users/00989e8a404546e6a1b1e4db9e06744e => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:23.911003 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-20758c50-cec9-48d6-a83e-b8974c62c51f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25e51d4495ee40bc86874411577cb6b8', 'request_id': 'req-20758c50-cec9-48d6-a83e-b8974c62c51f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.919037 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-20758c50-cec9-48d6-a83e-b8974c62c51f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25e51d4495ee40bc86874411577cb6b8', 'request_id': 'req-20758c50-cec9-48d6-a83e-b8974c62c51f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:23.925139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-20758c50-cec9-48d6-a83e-b8974c62c51f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25e51d4495ee40bc86874411577cb6b8', 'request_id': 'req-20758c50-cec9-48d6-a83e-b8974c62c51f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.007508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-20758c50-cec9-48d6-a83e-b8974c62c51f admin admin] The token cache is being invalidate because project 25e51d4495ee40bc86874411577cb6b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:24.007795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-20758c50-cec9-48d6-a83e-b8974c62c51f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.008415 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-20758c50-cec9-48d6-a83e-b8974c62c51f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.009903 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1084/2193] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:23 2022] DELETE /identity/v3/projects/25e51d4495ee40bc86874411577cb6b8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:24.015062 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc12b4-b490-45de-a530-9544c9f625ab None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:24.015373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc12b4-b490-45de-a530-9544c9f625ab None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:24.015646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc12b4-b490-45de-a530-9544c9f625ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:24.035146 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-9fbc12b4-b490-45de-a530-9544c9f625ab None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:24.036346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9fbc12b4-b490-45de-a530-9544c9f625ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:24.083844 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:24.107328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1110/2194] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:24.108594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708131fa-3adf-446a-9b61-87ee2ae00846 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:24.108880 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708131fa-3adf-446a-9b61-87ee2ae00846 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:24.109166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708131fa-3adf-446a-9b61-87ee2ae00846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:24.132222 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-708131fa-3adf-446a-9b61-87ee2ae00846 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:24.133495 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-708131fa-3adf-446a-9b61-87ee2ae00846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:24.141953 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1085/2195] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:24.147696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc12b4-b490-45de-a530-9544c9f625ab None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:24.158574 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc12b4-b490-45de-a530-9544c9f625ab None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:24.163434 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf5ec3-7e92-40cf-9c34-a2502b09c4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:24.166916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf5ec3-7e92-40cf-9c34-a2502b09c4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:24.166916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf5ec3-7e92-40cf-9c34-a2502b09c4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:24.166916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf5ec3-7e92-40cf-9c34-a2502b09c4a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:24.166916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf5ec3-7e92-40cf-9c34-a2502b09c4a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:24.166916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf5ec3-7e92-40cf-9c34-a2502b09c4a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:24.168021 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf5ec3-7e92-40cf-9c34-a2502b09c4a7 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:24.184833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1111/2196] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:58:24.186043 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708131fa-3adf-446a-9b61-87ee2ae00846 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:24.195753 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708131fa-3adf-446a-9b61-87ee2ae00846 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:24.200768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77bc31e-1a35-4fa4-9804-91405797ab99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:24.201370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bc31e-1a35-4fa4-9804-91405797ab99 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:24.201646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bc31e-1a35-4fa4-9804-91405797ab99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:24.201913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bc31e-1a35-4fa4-9804-91405797ab99 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:24.202542 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77bc31e-1a35-4fa4-9804-91405797ab99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:24.204575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77bc31e-1a35-4fa4-9804-91405797ab99 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:24.311941 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1086/2197] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:24 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:24.316009 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf5ec3-7e92-40cf-9c34-a2502b09c4a7 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:24.325761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf5ec3-7e92-40cf-9c34-a2502b09c4a7 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:24.339734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77bc31e-1a35-4fa4-9804-91405797ab99 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:24.341866 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a77bc31e-1a35-4fa4-9804-91405797ab99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:24.389388 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d434d96-335f-47f8-a140-6c06a045b651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:24.389992 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d434d96-335f-47f8-a140-6c06a045b651 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:24.390277 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d434d96-335f-47f8-a140-6c06a045b651 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:24.390543 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d434d96-335f-47f8-a140-6c06a045b651 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:24.391213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d434d96-335f-47f8-a140-6c06a045b651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:24.393130 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d434d96-335f-47f8-a140-6c06a045b651 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:24.414376 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77bc31e-1a35-4fa4-9804-91405797ab99 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:24.419113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c95b66-0cba-49c8-8634-b9c420b18cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:24.422931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c95b66-0cba-49c8-8634-b9c420b18cea admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:24.422931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c95b66-0cba-49c8-8634-b9c420b18cea admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:24.422931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c95b66-0cba-49c8-8634-b9c420b18cea admin admin] PATH_INFO: `/v3/projects/03ffcce6b6ef49b1a50a9f2ed99471e2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:24.422931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c95b66-0cba-49c8-8634-b9c420b18cea admin admin] RBAC: Authorizing `identity:delete_project(project_id=03ffcce6b6ef49b1a50a9f2ed99471e2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:24.423406 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c95b66-0cba-49c8-8634-b9c420b18cea admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:24.499251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1112/2198] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:24 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:24.503271 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d434d96-335f-47f8-a140-6c06a045b651 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:24.512608 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d434d96-335f-47f8-a140-6c06a045b651 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:24.517513 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad5d23-49b8-4f8f-95e5-5fd8fb852d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:24.518090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad5d23-49b8-4f8f-95e5-5fd8fb852d98 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:24.518362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad5d23-49b8-4f8f-95e5-5fd8fb852d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:24.518627 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad5d23-49b8-4f8f-95e5-5fd8fb852d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:24.519299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad5d23-49b8-4f8f-95e5-5fd8fb852d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:24.520965 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad5d23-49b8-4f8f-95e5-5fd8fb852d98 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:24.531972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1113/2199] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:24.535662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad5d23-49b8-4f8f-95e5-5fd8fb852d98 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:24.536169 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f6c95b66-0cba-49c8-8634-b9c420b18cea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03ffcce6b6ef49b1a50a9f2ed99471e2', 'request_id': 'req-f6c95b66-0cba-49c8-8634-b9c420b18cea'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.542995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f6c95b66-0cba-49c8-8634-b9c420b18cea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03ffcce6b6ef49b1a50a9f2ed99471e2', 'request_id': 'req-f6c95b66-0cba-49c8-8634-b9c420b18cea'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.545403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad5d23-49b8-4f8f-95e5-5fd8fb852d98 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:24.546940 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:24.549173 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f6c95b66-0cba-49c8-8634-b9c420b18cea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03ffcce6b6ef49b1a50a9f2ed99471e2', 'request_id': 'req-f6c95b66-0cba-49c8-8634-b9c420b18cea'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.600986 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fff8ae-544c-4a77-aab4-4475089c949d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:24.601644 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fff8ae-544c-4a77-aab4-4475089c949d admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:24.601923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fff8ae-544c-4a77-aab4-4475089c949d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:24.602193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fff8ae-544c-4a77-aab4-4475089c949d admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:24.602824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fff8ae-544c-4a77-aab4-4475089c949d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:24.604457 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fff8ae-544c-4a77-aab4-4475089c949d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:24.614257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1114/2200] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:24.618410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fff8ae-544c-4a77-aab4-4475089c949d admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:24.628224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fff8ae-544c-4a77-aab4-4475089c949d admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:24.633977 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e98e3-6c44-4215-a176-1695ac2ed355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:24.634832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e98e3-6c44-4215-a176-1695ac2ed355 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:24.635345 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e98e3-6c44-4215-a176-1695ac2ed355 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:24.635821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e98e3-6c44-4215-a176-1695ac2ed355 admin admin] PATH_INFO: `/v3/projects/453789ea3de34512885d8596d1aee6de/users/e8579725af034affb4f54b532b1f9313/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:24.637304 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f6c95b66-0cba-49c8-8634-b9c420b18cea admin admin] The token cache is being invalidate because project 03ffcce6b6ef49b1a50a9f2ed99471e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:24.637598 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f6c95b66-0cba-49c8-8634-b9c420b18cea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.638236 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f6c95b66-0cba-49c8-8634-b9c420b18cea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.641023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1087/2201] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:24 2022] DELETE /identity/v3/projects/03ffcce6b6ef49b1a50a9f2ed99471e2 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:24.644853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c95b66-0cba-49c8-8634-b9c420b18cea admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:24.646380 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f6c95b66-0cba-49c8-8634-b9c420b18cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:24.652700 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e98e3-6c44-4215-a176-1695ac2ed355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=453789ea3de34512885d8596d1aee6de, user_id=e8579725af034affb4f54b532b1f9313, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:24.655389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e98e3-6c44-4215-a176-1695ac2ed355 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:24.660606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c95b66-0cba-49c8-8634-b9c420b18cea admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:24.665741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a098514-9acf-4aa5-9c63-384764699261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:24.669848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a098514-9acf-4aa5-9c63-384764699261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:24.669848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a098514-9acf-4aa5-9c63-384764699261 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:24.669848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a098514-9acf-4aa5-9c63-384764699261 admin admin] PATH_INFO: `/v3/users/b9d32cd71e0c4feb874bd2256b7c7116` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:24.669848 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a098514-9acf-4aa5-9c63-384764699261 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9d32cd71e0c4feb874bd2256b7c7116)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:24.670174 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a098514-9acf-4aa5-9c63-384764699261 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:24.750444 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1115/2202] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:24 2022] PUT /identity/v3/projects/453789ea3de34512885d8596d1aee6de/users/e8579725af034affb4f54b532b1f9313/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:24.761849 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62472f-c7a2-4ab1-a312-879a15e35d04 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:24.762132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62472f-c7a2-4ab1-a312-879a15e35d04 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:24.762404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62472f-c7a2-4ab1-a312-879a15e35d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:24.790047 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-ff62472f-c7a2-4ab1-a312-879a15e35d04 None None] MFA Rules not processed for user `e8579725af034affb4f54b532b1f9313`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:24.791306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ff62472f-c7a2-4ab1-a312-879a15e35d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:24.804029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:24.854519 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1116/2203] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:24.859180 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62472f-c7a2-4ab1-a312-879a15e35d04 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:24.860449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ff62472f-c7a2-4ab1-a312-879a15e35d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:24.879511 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6a098514-9acf-4aa5-9c63-384764699261 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9d32cd71e0c4feb874bd2256b7c7116'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.887349 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6a098514-9acf-4aa5-9c63-384764699261 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9d32cd71e0c4feb874bd2256b7c7116'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.888067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6a098514-9acf-4aa5-9c63-384764699261 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9d32cd71e0c4feb874bd2256b7c7116'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.924109 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62472f-c7a2-4ab1-a312-879a15e35d04 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:24.928738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807d93b-27fb-4927-af9a-473b84226776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:24.929340 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807d93b-27fb-4927-af9a-473b84226776 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:24.929619 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807d93b-27fb-4927-af9a-473b84226776 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:24.929891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807d93b-27fb-4927-af9a-473b84226776 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:24.930514 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807d93b-27fb-4927-af9a-473b84226776 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:24.932587 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807d93b-27fb-4927-af9a-473b84226776 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:24.972945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6a098514-9acf-4aa5-9c63-384764699261 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9d32cd71e0c4feb874bd2256b7c7116'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.985711 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6a098514-9acf-4aa5-9c63-384764699261 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9d32cd71e0c4feb874bd2256b7c7116'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:24.992732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1088/2204] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:24 2022] DELETE /identity/v3/users/b9d32cd71e0c4feb874bd2256b7c7116 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:25.036042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1117/2205] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:24 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:25.039734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a098514-9acf-4aa5-9c63-384764699261 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:25.041833 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6a098514-9acf-4aa5-9c63-384764699261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:25.129748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a098514-9acf-4aa5-9c63-384764699261 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:25.186577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d533f-ffec-478c-a8f5-5375f1128efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:25.187215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d533f-ffec-478c-a8f5-5375f1128efb admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.187510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d533f-ffec-478c-a8f5-5375f1128efb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.187774 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d533f-ffec-478c-a8f5-5375f1128efb admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.188404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d533f-ffec-478c-a8f5-5375f1128efb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:25.190330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d533f-ffec-478c-a8f5-5375f1128efb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:25.202044 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=12,keystone:SELECT=83,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:25.300358 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1089/2206] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:25 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:25.300493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807d93b-27fb-4927-af9a-473b84226776 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:25.305631 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807d93b-27fb-4927-af9a-473b84226776 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:25.310689 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361f248-5081-4795-8286-14b9c699577e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:25.311328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361f248-5081-4795-8286-14b9c699577e admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.311801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361f248-5081-4795-8286-14b9c699577e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.312184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361f248-5081-4795-8286-14b9c699577e admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.312839 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361f248-5081-4795-8286-14b9c699577e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:25.314475 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361f248-5081-4795-8286-14b9c699577e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:25.325893 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1118/2207] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:25.330116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d533f-ffec-478c-a8f5-5375f1128efb admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:25.341685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d533f-ffec-478c-a8f5-5375f1128efb admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:25.346383 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0f5fa-68ed-491d-b071-1f623743d09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:25.347046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0f5fa-68ed-491d-b071-1f623743d09b admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.347415 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0f5fa-68ed-491d-b071-1f623743d09b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.347751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0f5fa-68ed-491d-b071-1f623743d09b admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.348465 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0f5fa-68ed-491d-b071-1f623743d09b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:25.350128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0f5fa-68ed-491d-b071-1f623743d09b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:25.363145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1090/2208] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:25.366029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361f248-5081-4795-8286-14b9c699577e admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:25.376552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361f248-5081-4795-8286-14b9c699577e admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:25.381358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4b8f1-d914-4dae-b867-2882a350e2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:25.381943 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4b8f1-d914-4dae-b867-2882a350e2bd admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.383148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4b8f1-d914-4dae-b867-2882a350e2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.385157 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4b8f1-d914-4dae-b867-2882a350e2bd admin admin] PATH_INFO: `/v3/projects/1bf364aeda6e4a57abf71b1828b16fcb/users/6672d70199994c6882cbe8d6d4cc7857/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.402184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0f5fa-68ed-491d-b071-1f623743d09b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:25.402862 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fcd0f5fa-68ed-491d-b071-1f623743d09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:25.404074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4b8f1-d914-4dae-b867-2882a350e2bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf364aeda6e4a57abf71b1828b16fcb, user_id=6672d70199994c6882cbe8d6d4cc7857, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:25.407011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4b8f1-d914-4dae-b867-2882a350e2bd admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:25.418841 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0f5fa-68ed-491d-b071-1f623743d09b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:25.423949 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09a8df-a091-41cb-b4d4-de9ba3af8bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:25.424684 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09a8df-a091-41cb-b4d4-de9ba3af8bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.425043 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09a8df-a091-41cb-b4d4-de9ba3af8bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.425231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09a8df-a091-41cb-b4d4-de9ba3af8bf7 admin admin] PATH_INFO: `/v3/projects/1c895d7bedd34896b55703c9e17c56f4` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.426414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09a8df-a091-41cb-b4d4-de9ba3af8bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c895d7bedd34896b55703c9e17c56f4)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:25.428354 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09a8df-a091-41cb-b4d4-de9ba3af8bf7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:25.488191 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1119/2209] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:25 2022] PUT /identity/v3/projects/1bf364aeda6e4a57abf71b1828b16fcb/users/6672d70199994c6882cbe8d6d4cc7857/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:25.498871 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae707615-ad5c-4ce2-9240-6fb39932d066 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.499203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae707615-ad5c-4ce2-9240-6fb39932d066 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.499482 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae707615-ad5c-4ce2-9240-6fb39932d066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.536213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-ae707615-ad5c-4ce2-9240-6fb39932d066 None None] MFA Rules not processed for user `6672d70199994c6882cbe8d6d4cc7857`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:25.537436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ae707615-ad5c-4ce2-9240-6fb39932d066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:25.547741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2c09a8df-a091-41cb-b4d4-de9ba3af8bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c895d7bedd34896b55703c9e17c56f4', 'request_id': 'req-2c09a8df-a091-41cb-b4d4-de9ba3af8bf7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.554380 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2c09a8df-a091-41cb-b4d4-de9ba3af8bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c895d7bedd34896b55703c9e17c56f4', 'request_id': 'req-2c09a8df-a091-41cb-b4d4-de9ba3af8bf7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.559952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2c09a8df-a091-41cb-b4d4-de9ba3af8bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c895d7bedd34896b55703c9e17c56f4', 'request_id': 'req-2c09a8df-a091-41cb-b4d4-de9ba3af8bf7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.594367 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:25.603093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1120/2210] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:25.607773 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae707615-ad5c-4ce2-9240-6fb39932d066 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:25.646702 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2c09a8df-a091-41cb-b4d4-de9ba3af8bf7 admin admin] The token cache is being invalidate because project 1c895d7bedd34896b55703c9e17c56f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:25.647086 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2c09a8df-a091-41cb-b4d4-de9ba3af8bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.648615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2c09a8df-a091-41cb-b4d4-de9ba3af8bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.650386 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1091/2211] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:25 2022] DELETE /identity/v3/projects/1c895d7bedd34896b55703c9e17c56f4 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:25.656206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b9de7-33f3-46aa-9d02-998402804eb7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.656486 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b9de7-33f3-46aa-9d02-998402804eb7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.656755 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b9de7-33f3-46aa-9d02-998402804eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.676205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b13b9de7-33f3-46aa-9d02-998402804eb7 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:25.677402 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b13b9de7-33f3-46aa-9d02-998402804eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:25.687294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae707615-ad5c-4ce2-9240-6fb39932d066 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:25.692002 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf730aea-f26b-417c-acbc-6dbf5fe26dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:25.692651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf730aea-f26b-417c-acbc-6dbf5fe26dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.696692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf730aea-f26b-417c-acbc-6dbf5fe26dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.696692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf730aea-f26b-417c-acbc-6dbf5fe26dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.696692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf730aea-f26b-417c-acbc-6dbf5fe26dd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:25.696692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf730aea-f26b-417c-acbc-6dbf5fe26dd4 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:25.697487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1092/2212] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:25.706056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c63945-3a1e-437f-9eb0-9438ec8dadc5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.706346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c63945-3a1e-437f-9eb0-9438ec8dadc5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.706618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c63945-3a1e-437f-9eb0-9438ec8dadc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.727391 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-27c63945-3a1e-437f-9eb0-9438ec8dadc5 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:25.728597 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-27c63945-3a1e-437f-9eb0-9438ec8dadc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:25.736601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1093/2213] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:25.740646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c63945-3a1e-437f-9eb0-9438ec8dadc5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:25.750129 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c63945-3a1e-437f-9eb0-9438ec8dadc5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:25.760034 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e6239-fd7c-4aed-b68d-bf18f2bcc396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:25.760034 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e6239-fd7c-4aed-b68d-bf18f2bcc396 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.760034 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e6239-fd7c-4aed-b68d-bf18f2bcc396 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.760034 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e6239-fd7c-4aed-b68d-bf18f2bcc396 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.760988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e6239-fd7c-4aed-b68d-bf18f2bcc396 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:25.760988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e6239-fd7c-4aed-b68d-bf18f2bcc396 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:25.760988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e6239-fd7c-4aed-b68d-bf18f2bcc396 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:25.772198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1094/2214] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:58:25.776424 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e6239-fd7c-4aed-b68d-bf18f2bcc396 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:25.786194 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e6239-fd7c-4aed-b68d-bf18f2bcc396 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:25.790961 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0035d2d-63e1-4f58-91ab-c193be7c28af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:25.791582 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0035d2d-63e1-4f58-91ab-c193be7c28af admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.791858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0035d2d-63e1-4f58-91ab-c193be7c28af admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.792158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0035d2d-63e1-4f58-91ab-c193be7c28af admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.792781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0035d2d-63e1-4f58-91ab-c193be7c28af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:25.794676 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0035d2d-63e1-4f58-91ab-c193be7c28af admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:25.806497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1121/2215] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:25 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:25.810250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf730aea-f26b-417c-acbc-6dbf5fe26dd4 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:25.812106 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-bf730aea-f26b-417c-acbc-6dbf5fe26dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:25.875117 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf730aea-f26b-417c-acbc-6dbf5fe26dd4 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:25.879987 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6388333-3c02-4c8d-8bdf-d2f9b27f7507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:25.880580 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6388333-3c02-4c8d-8bdf-d2f9b27f7507 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.880855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6388333-3c02-4c8d-8bdf-d2f9b27f7507 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.881124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6388333-3c02-4c8d-8bdf-d2f9b27f7507 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.881741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6388333-3c02-4c8d-8bdf-d2f9b27f7507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:25.884326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6388333-3c02-4c8d-8bdf-d2f9b27f7507 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:25.955367 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1095/2216] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:25 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:25.956249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0035d2d-63e1-4f58-91ab-c193be7c28af admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:25.979642 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1122/2217] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:25 2022] POST /identity/v3/users => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:25.985326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6388333-3c02-4c8d-8bdf-d2f9b27f7507 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:25.995756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6388333-3c02-4c8d-8bdf-d2f9b27f7507 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:26.021849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0035d2d-63e1-4f58-91ab-c193be7c28af admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:26.027348 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefab027-9244-44ab-88f4-7ec31525a93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:26.027953 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefab027-9244-44ab-88f4-7ec31525a93f admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:26.028233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefab027-9244-44ab-88f4-7ec31525a93f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:26.028535 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefab027-9244-44ab-88f4-7ec31525a93f admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:26.029185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefab027-9244-44ab-88f4-7ec31525a93f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:26.031215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefab027-9244-44ab-88f4-7ec31525a93f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:26.052196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f149a-acd0-4292-a5a2-ea1e89c0c6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:26.052951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f149a-acd0-4292-a5a2-ea1e89c0c6be admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:26.053234 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f149a-acd0-4292-a5a2-ea1e89c0c6be admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:26.053504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f149a-acd0-4292-a5a2-ea1e89c0c6be admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:26.054661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f149a-acd0-4292-a5a2-ea1e89c0c6be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:26.056441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f149a-acd0-4292-a5a2-ea1e89c0c6be admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:26.066232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1123/2218] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:26.071093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f149a-acd0-4292-a5a2-ea1e89c0c6be admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:26.080983 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f149a-acd0-4292-a5a2-ea1e89c0c6be admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:26.085690 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdab06-6f76-4212-8014-518fbf75424f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:26.086306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdab06-6f76-4212-8014-518fbf75424f admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:26.086585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdab06-6f76-4212-8014-518fbf75424f admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:26.086859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdab06-6f76-4212-8014-518fbf75424f admin admin] PATH_INFO: `/v3/projects/d86302db7f1e4f418dff4ab9223cd920/users/c971cafede0f4454a3fa53998a1d310b/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:26.089074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:26.102201 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdab06-6f76-4212-8014-518fbf75424f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d86302db7f1e4f418dff4ab9223cd920, user_id=c971cafede0f4454a3fa53998a1d310b, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:26.105303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdab06-6f76-4212-8014-518fbf75424f admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:26.157061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1096/2219] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:25 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:26.161224 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefab027-9244-44ab-88f4-7ec31525a93f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:26.171455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefab027-9244-44ab-88f4-7ec31525a93f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:26.176048 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd762b5-8685-41b9-8c00-732bf7909d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:26.176621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd762b5-8685-41b9-8c00-732bf7909d41 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:26.176895 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd762b5-8685-41b9-8c00-732bf7909d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:26.177203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd762b5-8685-41b9-8c00-732bf7909d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:26.177856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd762b5-8685-41b9-8c00-732bf7909d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:26.180143 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd762b5-8685-41b9-8c00-732bf7909d41 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:26.182136 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:26.190550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1097/2220] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:26.190993 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1124/2221] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:26 2022] PUT /identity/v3/projects/d86302db7f1e4f418dff4ab9223cd920/users/c971cafede0f4454a3fa53998a1d310b/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:26.194938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd762b5-8685-41b9-8c00-732bf7909d41 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:26.195236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdab06-6f76-4212-8014-518fbf75424f admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:26.204631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd762b5-8685-41b9-8c00-732bf7909d41 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:26.257608 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdab06-6f76-4212-8014-518fbf75424f admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:26.260543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e43fb-c8fb-422a-b7e4-48732b3ff93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:26.261254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e43fb-c8fb-422a-b7e4-48732b3ff93f admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:26.261488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e43fb-c8fb-422a-b7e4-48732b3ff93f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:26.261759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e43fb-c8fb-422a-b7e4-48732b3ff93f admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:26.262403 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e43fb-c8fb-422a-b7e4-48732b3ff93f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:26.262513 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af87f6-4d88-48a7-a6e7-d1ed26382149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:26.263270 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af87f6-4d88-48a7-a6e7-d1ed26382149 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:26.263487 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af87f6-4d88-48a7-a6e7-d1ed26382149 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:26.263762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af87f6-4d88-48a7-a6e7-d1ed26382149 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:26.264258 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e43fb-c8fb-422a-b7e4-48732b3ff93f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:26.264666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af87f6-4d88-48a7-a6e7-d1ed26382149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:26.266234 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af87f6-4d88-48a7-a6e7-d1ed26382149 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:26.275761 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1098/2222] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:26.277334 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1125/2223] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:26.279773 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e43fb-c8fb-422a-b7e4-48732b3ff93f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:26.280472 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af87f6-4d88-48a7-a6e7-d1ed26382149 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:26.290316 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e43fb-c8fb-422a-b7e4-48732b3ff93f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:26.290824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af87f6-4d88-48a7-a6e7-d1ed26382149 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:26.295336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672dd52-a7ce-466c-932c-4deefb0cc03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:26.298829 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672dd52-a7ce-466c-932c-4deefb0cc03c admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:26.298940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec74e3-49de-4c1d-9964-8f4bbc4a7e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:26.299048 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672dd52-a7ce-466c-932c-4deefb0cc03c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:26.299048 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672dd52-a7ce-466c-932c-4deefb0cc03c admin admin] PATH_INFO: `/v3/projects/d86302db7f1e4f418dff4ab9223cd920/users/c971cafede0f4454a3fa53998a1d310b/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:26.299224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec74e3-49de-4c1d-9964-8f4bbc4a7e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:26.299224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec74e3-49de-4c1d-9964-8f4bbc4a7e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:26.299224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec74e3-49de-4c1d-9964-8f4bbc4a7e7c admin admin] PATH_INFO: `/v3/projects/f60aa51efc8a4c00a10b987fa3465add/users/3fc014703e0b45eb99ed96e9d19a6306/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:26.299423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672dd52-a7ce-466c-932c-4deefb0cc03c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d86302db7f1e4f418dff4ab9223cd920, user_id=c971cafede0f4454a3fa53998a1d310b, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:26.301172 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672dd52-a7ce-466c-932c-4deefb0cc03c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:26.312743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec74e3-49de-4c1d-9964-8f4bbc4a7e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f60aa51efc8a4c00a10b987fa3465add, user_id=3fc014703e0b45eb99ed96e9d19a6306, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:26.313052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1099/2224] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:26 2022] PUT /identity/v3/projects/d86302db7f1e4f418dff4ab9223cd920/users/c971cafede0f4454a3fa53998a1d310b/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:26.315459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec74e3-49de-4c1d-9964-8f4bbc4a7e7c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:26.323202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7b69b-8ce3-4ef9-bf58-ddd825a031d1 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:26.323482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7b69b-8ce3-4ef9-bf58-ddd825a031d1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:26.323750 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7b69b-8ce3-4ef9-bf58-ddd825a031d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:26.351632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b2b7b69b-8ce3-4ef9-bf58-ddd825a031d1 None None] MFA Rules not processed for user `c971cafede0f4454a3fa53998a1d310b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:26.352824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b2b7b69b-8ce3-4ef9-bf58-ddd825a031d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:26.374173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1126/2225] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:26 2022] PUT /identity/v3/projects/f60aa51efc8a4c00a10b987fa3465add/users/3fc014703e0b45eb99ed96e9d19a6306/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:26.379262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fd670-d6d0-439c-838b-42fbb581c9b1 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:26.379550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fd670-d6d0-439c-838b-42fbb581c9b1 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:26.379822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fd670-d6d0-439c-838b-42fbb581c9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:26.398376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-529fd670-d6d0-439c-838b-42fbb581c9b1 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:26.403581 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-529fd670-d6d0-439c-838b-42fbb581c9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:26.421654 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1100/2226] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:26.434205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7b69b-8ce3-4ef9-bf58-ddd825a031d1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:26.435638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b2b7b69b-8ce3-4ef9-bf58-ddd825a031d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:26.462081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1127/2227] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:26.472010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529fd670-d6d0-439c-838b-42fbb581c9b1 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:26.473514 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-529fd670-d6d0-439c-838b-42fbb581c9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:26.494432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7b69b-8ce3-4ef9-bf58-ddd825a031d1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:26.513829 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:26.536477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529fd670-d6d0-439c-838b-42fbb581c9b1 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:26.540995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668fc4f-28da-4c3b-91b5-c87c9cc30884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:26.541462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668fc4f-28da-4c3b-91b5-c87c9cc30884 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:26.541814 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668fc4f-28da-4c3b-91b5-c87c9cc30884 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:26.542116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668fc4f-28da-4c3b-91b5-c87c9cc30884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:26.548540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c674c73-49b0-4ffa-89c6-827250d2fdd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:26.549151 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c674c73-49b0-4ffa-89c6-827250d2fdd5 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:26.549428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c674c73-49b0-4ffa-89c6-827250d2fdd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:26.549706 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c674c73-49b0-4ffa-89c6-827250d2fdd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:26.551587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1c674c73-49b0-4ffa-89c6-827250d2fdd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:26.554052 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668fc4f-28da-4c3b-91b5-c87c9cc30884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:26.555739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668fc4f-28da-4c3b-91b5-c87c9cc30884 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:26.560298 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:26.567309 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1128/2228] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:26.613628 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c674c73-49b0-4ffa-89c6-827250d2fdd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:26.615079 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c674c73-49b0-4ffa-89c6-827250d2fdd5 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:26.624805 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1101/2229] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:58:28.122803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64ffde-8e66-4a4f-9741-ebcab811480e None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:28.123352 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64ffde-8e66-4a4f-9741-ebcab811480e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:28.123445 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64ffde-8e66-4a4f-9741-ebcab811480e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:28.152156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-6f64ffde-8e66-4a4f-9741-ebcab811480e None None] MFA Rules not processed for user `3fc014703e0b45eb99ed96e9d19a6306`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:28.153278 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6f64ffde-8e66-4a4f-9741-ebcab811480e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:28.245362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1129/2230] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:28.249946 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c674c73-49b0-4ffa-89c6-827250d2fdd5 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:28.267208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c674c73-49b0-4ffa-89c6-827250d2fdd5 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:28.272120 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c141661-035b-40ed-9c41-f88ad699244a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:28.273306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c141661-035b-40ed-9c41-f88ad699244a admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:28.273587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c141661-035b-40ed-9c41-f88ad699244a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:28.273856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c141661-035b-40ed-9c41-f88ad699244a admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:28.274668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c141661-035b-40ed-9c41-f88ad699244a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:28.276748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c141661-035b-40ed-9c41-f88ad699244a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:28.343694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1102/2231] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:28 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:28.344925 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64ffde-8e66-4a4f-9741-ebcab811480e None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:28.357998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64ffde-8e66-4a4f-9741-ebcab811480e None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:28.429123 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220c990-330e-41e4-935a-d735a1f70149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:28.429733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220c990-330e-41e4-935a-d735a1f70149 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:28.430018 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220c990-330e-41e4-935a-d735a1f70149 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:28.430288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220c990-330e-41e4-935a-d735a1f70149 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:28.430933 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220c990-330e-41e4-935a-d735a1f70149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:28.433064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220c990-330e-41e4-935a-d735a1f70149 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:28.557184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1130/2232] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:28 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:28.563702 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c141661-035b-40ed-9c41-f88ad699244a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:28.575614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c141661-035b-40ed-9c41-f88ad699244a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:28.581277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadec67-0d1b-493c-87a3-962c89baf265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:28.584233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadec67-0d1b-493c-87a3-962c89baf265 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:28.584233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadec67-0d1b-493c-87a3-962c89baf265 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:28.584233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadec67-0d1b-493c-87a3-962c89baf265 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:28.584233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dadec67-0d1b-493c-87a3-962c89baf265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:28.584850 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dadec67-0d1b-493c-87a3-962c89baf265 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:28.597058 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1103/2233] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:28.601193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220c990-330e-41e4-935a-d735a1f70149 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:28.611613 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220c990-330e-41e4-935a-d735a1f70149 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:28.616321 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968c725-a7f3-4fbb-bd82-ac9953ac742a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:28.616993 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968c725-a7f3-4fbb-bd82-ac9953ac742a admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:28.617272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968c725-a7f3-4fbb-bd82-ac9953ac742a admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:28.617828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968c725-a7f3-4fbb-bd82-ac9953ac742a admin admin] PATH_INFO: `/v3/projects/1748c8178cf84b7b939a6aba4530fa3e/users/826ce40c72c646e7bd2ef2fd377b529f/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:28.708628 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadec67-0d1b-493c-87a3-962c89baf265 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:28.757973 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968c725-a7f3-4fbb-bd82-ac9953ac742a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1748c8178cf84b7b939a6aba4530fa3e, user_id=826ce40c72c646e7bd2ef2fd377b529f, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:28.760806 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968c725-a7f3-4fbb-bd82-ac9953ac742a admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:28.768728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadec67-0d1b-493c-87a3-962c89baf265 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:28.772688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfd9b3-5c7c-4e81-b1d5-942df1019a2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:28.773296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfd9b3-5c7c-4e81-b1d5-942df1019a2a service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:28.773572 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfd9b3-5c7c-4e81-b1d5-942df1019a2a service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:28.773837 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfd9b3-5c7c-4e81-b1d5-942df1019a2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:28.775682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-dbbfd9b3-5c7c-4e81-b1d5-942df1019a2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:28.833807 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfd9b3-5c7c-4e81-b1d5-942df1019a2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:28.835264 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfd9b3-5c7c-4e81-b1d5-942df1019a2a service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:28.838716 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:28.844945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1104/2234] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:58:28.848117 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1131/2235] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:28 2022] PUT /identity/v3/projects/1748c8178cf84b7b939a6aba4530fa3e/users/826ce40c72c646e7bd2ef2fd377b529f/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:28.851999 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfd9b3-5c7c-4e81-b1d5-942df1019a2a service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:28.862837 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfd9b3-5c7c-4e81-b1d5-942df1019a2a service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:28.928772 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50ef74-6a50-4938-a034-39ae692b6c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:28.930340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50ef74-6a50-4938-a034-39ae692b6c0a admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:28.930340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50ef74-6a50-4938-a034-39ae692b6c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:28.930655 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50ef74-6a50-4938-a034-39ae692b6c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:28.931296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50ef74-6a50-4938-a034-39ae692b6c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:28.932953 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50ef74-6a50-4938-a034-39ae692b6c0a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:28.944122 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1105/2236] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:28.950992 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968c725-a7f3-4fbb-bd82-ac9953ac742a admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:28.963687 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968c725-a7f3-4fbb-bd82-ac9953ac742a admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:28.968394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdb5c4-a31a-4104-91ac-b4e24cbeaca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:28.969029 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdb5c4-a31a-4104-91ac-b4e24cbeaca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:28.972926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdb5c4-a31a-4104-91ac-b4e24cbeaca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:28.972926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdb5c4-a31a-4104-91ac-b4e24cbeaca1 admin admin] PATH_INFO: `/v3/projects/1748c8178cf84b7b939a6aba4530fa3e/users/826ce40c72c646e7bd2ef2fd377b529f/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:28.972926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdb5c4-a31a-4104-91ac-b4e24cbeaca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1748c8178cf84b7b939a6aba4530fa3e, user_id=826ce40c72c646e7bd2ef2fd377b529f, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:28.974110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdb5c4-a31a-4104-91ac-b4e24cbeaca1 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:28.984484 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1132/2237] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:28 2022] PUT /identity/v3/projects/1748c8178cf84b7b939a6aba4530fa3e/users/826ce40c72c646e7bd2ef2fd377b529f/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:29.736377 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50ef74-6a50-4938-a034-39ae692b6c0a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:29.738139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5e50ef74-6a50-4938-a034-39ae692b6c0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:29.751701 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50ef74-6a50-4938-a034-39ae692b6c0a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:29.812594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a154da-b480-4c9e-ab10-52131a5c3967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:29.813225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a154da-b480-4c9e-ab10-52131a5c3967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:29.813501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a154da-b480-4c9e-ab10-52131a5c3967 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:29.813775 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a154da-b480-4c9e-ab10-52131a5c3967 admin admin] PATH_INFO: `/v3/users/66cd4f38ca4e4826a3a0d108a840530e` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:29.814886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a154da-b480-4c9e-ab10-52131a5c3967 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66cd4f38ca4e4826a3a0d108a840530e)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:29.817169 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a154da-b480-4c9e-ab10-52131a5c3967 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:30.050834 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-49a154da-b480-4c9e-ab10-52131a5c3967 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66cd4f38ca4e4826a3a0d108a840530e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:30.058070 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-49a154da-b480-4c9e-ab10-52131a5c3967 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66cd4f38ca4e4826a3a0d108a840530e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:30.058741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-49a154da-b480-4c9e-ab10-52131a5c3967 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66cd4f38ca4e4826a3a0d108a840530e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:30.132964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-49a154da-b480-4c9e-ab10-52131a5c3967 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66cd4f38ca4e4826a3a0d108a840530e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:30.150173 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-49a154da-b480-4c9e-ab10-52131a5c3967 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66cd4f38ca4e4826a3a0d108a840530e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:30.152010 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1106/2238] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:29 2022] DELETE /identity/v3/users/66cd4f38ca4e4826a3a0d108a840530e => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:30.156757 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f87c3-2695-479b-a6c5-fd8df8560fd9 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:30.157215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f87c3-2695-479b-a6c5-fd8df8560fd9 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:30.157331 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f87c3-2695-479b-a6c5-fd8df8560fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:30.176024 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-6c1f87c3-2695-479b-a6c5-fd8df8560fd9 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:30.181410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6c1f87c3-2695-479b-a6c5-fd8df8560fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:30.211271 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:30.244402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1133/2239] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:30.255277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a154da-b480-4c9e-ab10-52131a5c3967 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:30.258033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-49a154da-b480-4c9e-ab10-52131a5c3967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:30.275573 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:30.319584 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a154da-b480-4c9e-ab10-52131a5c3967 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:30.324277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b752b-27c6-48ae-aa88-461a0171caa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:30.324908 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b752b-27c6-48ae-aa88-461a0171caa8 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:30.325203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b752b-27c6-48ae-aa88-461a0171caa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:30.325464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b752b-27c6-48ae-aa88-461a0171caa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:30.335482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b752b-27c6-48ae-aa88-461a0171caa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:30.337059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b752b-27c6-48ae-aa88-461a0171caa8 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:30.346676 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1107/2240] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:30.635786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83998ce5-5339-4e34-a54c-a169470f2b66 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:30.635786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83998ce5-5339-4e34-a54c-a169470f2b66 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:30.635786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83998ce5-5339-4e34-a54c-a169470f2b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:30.665694 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-83998ce5-5339-4e34-a54c-a169470f2b66 None None] MFA Rules not processed for user `826ce40c72c646e7bd2ef2fd377b529f`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:30.666986 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-83998ce5-5339-4e34-a54c-a169470f2b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:30.749990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1134/2241] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:30.768226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b752b-27c6-48ae-aa88-461a0171caa8 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:30.773080 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2f4b752b-27c6-48ae-aa88-461a0171caa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:30.798054 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83998ce5-5339-4e34-a54c-a169470f2b66 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:30.799956 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-83998ce5-5339-4e34-a54c-a169470f2b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:30.832198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b752b-27c6-48ae-aa88-461a0171caa8 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:30.836345 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1c2f3-765c-40d8-a032-77633576f86e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:30.837061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1c2f3-765c-40d8-a032-77633576f86e service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:30.837404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1c2f3-765c-40d8-a032-77633576f86e service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:30.837742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1c2f3-765c-40d8-a032-77633576f86e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:30.839564 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-10a1c2f3-765c-40d8-a032-77633576f86e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:30.857660 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83998ce5-5339-4e34-a54c-a169470f2b66 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:30.861319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679511c8-3b08-4065-8e30-b5135ccd89a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:30.861846 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679511c8-3b08-4065-8e30-b5135ccd89a4 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:30.862117 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679511c8-3b08-4065-8e30-b5135ccd89a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:30.862379 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679511c8-3b08-4065-8e30-b5135ccd89a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:30.864185 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-679511c8-3b08-4065-8e30-b5135ccd89a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:30.910499 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1c2f3-765c-40d8-a032-77633576f86e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:30.914730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1c2f3-765c-40d8-a032-77633576f86e service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:30.930334 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1108/2242] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:30.931820 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1c2f3-765c-40d8-a032-77633576f86e service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:30.933220 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-10a1c2f3-765c-40d8-a032-77633576f86e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:30.946753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679511c8-3b08-4065-8e30-b5135ccd89a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:30.948292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679511c8-3b08-4065-8e30-b5135ccd89a4 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:30.953887 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1c2f3-765c-40d8-a032-77633576f86e service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:30.958807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1135/2243] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:58:30.959009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1107e26-eca5-444d-823e-d4df99ba41e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:30.959897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1107e26-eca5-444d-823e-d4df99ba41e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:30.960756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1107e26-eca5-444d-823e-d4df99ba41e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:30.961009 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1107e26-eca5-444d-823e-d4df99ba41e4 admin admin] PATH_INFO: `/v3/projects/ba5604a408ba402c9a0bfe2b2d78955b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:30.967975 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1107e26-eca5-444d-823e-d4df99ba41e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba5604a408ba402c9a0bfe2b2d78955b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:30.971164 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1107e26-eca5-444d-823e-d4df99ba41e4 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:31.071376 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e1107e26-eca5-444d-823e-d4df99ba41e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba5604a408ba402c9a0bfe2b2d78955b', 'request_id': 'req-e1107e26-eca5-444d-823e-d4df99ba41e4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:31.077823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e1107e26-eca5-444d-823e-d4df99ba41e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba5604a408ba402c9a0bfe2b2d78955b', 'request_id': 'req-e1107e26-eca5-444d-823e-d4df99ba41e4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:31.085629 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e1107e26-eca5-444d-823e-d4df99ba41e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba5604a408ba402c9a0bfe2b2d78955b', 'request_id': 'req-e1107e26-eca5-444d-823e-d4df99ba41e4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:31.172889 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e1107e26-eca5-444d-823e-d4df99ba41e4 admin admin] The token cache is being invalidate because project ba5604a408ba402c9a0bfe2b2d78955b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:31.173329 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e1107e26-eca5-444d-823e-d4df99ba41e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:31.173891 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-e1107e26-eca5-444d-823e-d4df99ba41e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:31.175599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1109/2244] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:30 2022] DELETE /identity/v3/projects/ba5604a408ba402c9a0bfe2b2d78955b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:31.180918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a3abb-8fe4-41d3-b742-8acb6e3806eb None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:31.181335 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a3abb-8fe4-41d3-b742-8acb6e3806eb None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:31.181410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a3abb-8fe4-41d3-b742-8acb6e3806eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:31.205881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-ae2a3abb-8fe4-41d3-b742-8acb6e3806eb None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:31.207257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ae2a3abb-8fe4-41d3-b742-8acb6e3806eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:31.220233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:31.280274 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1136/2245] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:31.292714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0afd6a-4f9a-4496-8a18-75605330ad84 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:31.293064 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0afd6a-4f9a-4496-8a18-75605330ad84 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:31.293343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0afd6a-4f9a-4496-8a18-75605330ad84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:31.329221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bf0afd6a-4f9a-4496-8a18-75605330ad84 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:31.330543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bf0afd6a-4f9a-4496-8a18-75605330ad84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:31.339945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1110/2246] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:31.347318 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a3abb-8fe4-41d3-b742-8acb6e3806eb None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:31.355093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a3abb-8fe4-41d3-b742-8acb6e3806eb None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:31.359923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37874466-d750-4fb5-846a-df0d552f4cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:31.360523 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37874466-d750-4fb5-846a-df0d552f4cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:31.361493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37874466-d750-4fb5-846a-df0d552f4cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:31.361751 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37874466-d750-4fb5-846a-df0d552f4cf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:31.362515 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37874466-d750-4fb5-846a-df0d552f4cf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:31.362705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37874466-d750-4fb5-846a-df0d552f4cf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:31.364536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37874466-d750-4fb5-846a-df0d552f4cf4 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:31.376510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1137/2247] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:58:31.381177 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0afd6a-4f9a-4496-8a18-75605330ad84 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:31.391584 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0afd6a-4f9a-4496-8a18-75605330ad84 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:31.397428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a225453-9d02-404b-a867-33d55bb43810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:31.398610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a225453-9d02-404b-a867-33d55bb43810 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:31.398892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a225453-9d02-404b-a867-33d55bb43810 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:31.399198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a225453-9d02-404b-a867-33d55bb43810 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:31.399968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a225453-9d02-404b-a867-33d55bb43810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:31.401907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a225453-9d02-404b-a867-33d55bb43810 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:31.441924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:31.563489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1111/2248] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:31 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:31.569269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37874466-d750-4fb5-846a-df0d552f4cf4 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:31.580141 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37874466-d750-4fb5-846a-df0d552f4cf4 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:31.643521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f2083-252b-4504-ae54-41a8667359f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:31.643877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f2083-252b-4504-ae54-41a8667359f8 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:31.644149 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f2083-252b-4504-ae54-41a8667359f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:31.644416 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f2083-252b-4504-ae54-41a8667359f8 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:31.645053 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f2083-252b-4504-ae54-41a8667359f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:31.646948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f2083-252b-4504-ae54-41a8667359f8 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:31.778588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1138/2249] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:31 2022] POST /identity/v3/users => generated 288 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:31.782653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a225453-9d02-404b-a867-33d55bb43810 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:31.792908 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a225453-9d02-404b-a867-33d55bb43810 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:31.797450 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65019a7-c6ca-4d6b-a680-e5c388dbce2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:31.798036 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65019a7-c6ca-4d6b-a680-e5c388dbce2d admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:31.798311 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65019a7-c6ca-4d6b-a680-e5c388dbce2d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:31.798576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65019a7-c6ca-4d6b-a680-e5c388dbce2d admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:31.799333 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65019a7-c6ca-4d6b-a680-e5c388dbce2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:31.801471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65019a7-c6ca-4d6b-a680-e5c388dbce2d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:31.812521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1112/2250] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:31.817247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f2083-252b-4504-ae54-41a8667359f8 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:31.828783 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f2083-252b-4504-ae54-41a8667359f8 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:31.833343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f348f-c256-411d-8de8-9b7970f5d615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:31.836974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f348f-c256-411d-8de8-9b7970f5d615 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:31.836974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f348f-c256-411d-8de8-9b7970f5d615 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:31.836974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f348f-c256-411d-8de8-9b7970f5d615 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:31.836974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323f348f-c256-411d-8de8-9b7970f5d615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:31.837981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323f348f-c256-411d-8de8-9b7970f5d615 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:31.851037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1139/2251] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:31.855375 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65019a7-c6ca-4d6b-a680-e5c388dbce2d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:31.865303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65019a7-c6ca-4d6b-a680-e5c388dbce2d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:31.870296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ee609-435b-490f-986d-5841c43a5fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:31.870938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ee609-435b-490f-986d-5841c43a5fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:31.871251 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ee609-435b-490f-986d-5841c43a5fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:31.871527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ee609-435b-490f-986d-5841c43a5fb8 admin admin] PATH_INFO: `/v3/projects/aa8c3314237d46ff89a34b0064ad2714/users/ed6ea1fff22445468245117de885c0eb/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:31.886794 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ee609-435b-490f-986d-5841c43a5fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8c3314237d46ff89a34b0064ad2714, user_id=ed6ea1fff22445468245117de885c0eb, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:31.889475 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ee609-435b-490f-986d-5841c43a5fb8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:31.989978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1113/2252] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:31 2022] PUT /identity/v3/projects/aa8c3314237d46ff89a34b0064ad2714/users/ed6ea1fff22445468245117de885c0eb/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:32.000654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585c689-74eb-4036-a4ee-d50d5ce3bbb8 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:32.000939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585c689-74eb-4036-a4ee-d50d5ce3bbb8 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:32.001212 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585c689-74eb-4036-a4ee-d50d5ce3bbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:32.031461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e585c689-74eb-4036-a4ee-d50d5ce3bbb8 None None] MFA Rules not processed for user `ed6ea1fff22445468245117de885c0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:32.032701 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e585c689-74eb-4036-a4ee-d50d5ce3bbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:32.096732 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1140/2253] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:32.101787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ee609-435b-490f-986d-5841c43a5fb8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:32.162093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ee609-435b-490f-986d-5841c43a5fb8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:32.167258 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b3528-ac7c-4b1a-a92c-a8a9e9294100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:32.167858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b3528-ac7c-4b1a-a92c-a8a9e9294100 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:32.168139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b3528-ac7c-4b1a-a92c-a8a9e9294100 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:32.168411 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b3528-ac7c-4b1a-a92c-a8a9e9294100 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:32.169033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b3528-ac7c-4b1a-a92c-a8a9e9294100 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:32.170926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b3528-ac7c-4b1a-a92c-a8a9e9294100 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:32.289659 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1114/2254] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:32 2022] POST /identity/v3/projects => generated 375 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:32.292413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585c689-74eb-4036-a4ee-d50d5ce3bbb8 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:32.303373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585c689-74eb-4036-a4ee-d50d5ce3bbb8 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:32.312206 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:32.360773 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91515431-ae14-49c0-a200-34f12429ae76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:32.361373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91515431-ae14-49c0-a200-34f12429ae76 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:32.361653 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91515431-ae14-49c0-a200-34f12429ae76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:32.361927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91515431-ae14-49c0-a200-34f12429ae76 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:32.362554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91515431-ae14-49c0-a200-34f12429ae76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:32.364508 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91515431-ae14-49c0-a200-34f12429ae76 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:32.667416 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1141/2255] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:32 2022] POST /identity/v3/users => generated 288 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:32.671460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b3528-ac7c-4b1a-a92c-a8a9e9294100 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:32.682338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b3528-ac7c-4b1a-a92c-a8a9e9294100 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:32.687126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5252585-f8cc-4e21-8587-fd676d371c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:32.687726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5252585-f8cc-4e21-8587-fd676d371c5b admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:32.687999 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5252585-f8cc-4e21-8587-fd676d371c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:32.688264 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5252585-f8cc-4e21-8587-fd676d371c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:32.688912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5252585-f8cc-4e21-8587-fd676d371c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:32.690536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5252585-f8cc-4e21-8587-fd676d371c5b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:32.701928 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1115/2256] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:32.705497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91515431-ae14-49c0-a200-34f12429ae76 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:32.715970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91515431-ae14-49c0-a200-34f12429ae76 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:32.720639 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25354de0-ac0b-4d02-a144-99f3612a87d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:32.721242 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25354de0-ac0b-4d02-a144-99f3612a87d3 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:32.721522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25354de0-ac0b-4d02-a144-99f3612a87d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:32.721790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25354de0-ac0b-4d02-a144-99f3612a87d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:32.722432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25354de0-ac0b-4d02-a144-99f3612a87d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:32.724037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25354de0-ac0b-4d02-a144-99f3612a87d3 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:32.735007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1142/2257] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:32.739071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5252585-f8cc-4e21-8587-fd676d371c5b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:32.749019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5252585-f8cc-4e21-8587-fd676d371c5b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:32.753595 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9860424-4cdc-4795-8073-25956acfddd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:32.754626 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9860424-4cdc-4795-8073-25956acfddd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:32.754626 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9860424-4cdc-4795-8073-25956acfddd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:32.754862 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9860424-4cdc-4795-8073-25956acfddd8 admin admin] PATH_INFO: `/v3/projects/00ec3dc042e645bd823869970c51903e/users/5a8c165029d547c184f642480d3e0f95/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:32.768872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9860424-4cdc-4795-8073-25956acfddd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00ec3dc042e645bd823869970c51903e, user_id=5a8c165029d547c184f642480d3e0f95, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:32.771560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9860424-4cdc-4795-8073-25956acfddd8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:32.839493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25354de0-ac0b-4d02-a144-99f3612a87d3 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:32.841189 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-25354de0-ac0b-4d02-a144-99f3612a87d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:32.870790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1116/2258] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:32 2022] PUT /identity/v3/projects/00ec3dc042e645bd823869970c51903e/users/5a8c165029d547c184f642480d3e0f95/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:32.882746 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b36055-c72e-4a85-ab26-0498b0277f99 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:32.883040 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b36055-c72e-4a85-ab26-0498b0277f99 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:32.883579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b36055-c72e-4a85-ab26-0498b0277f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:32.901633 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25354de0-ac0b-4d02-a144-99f3612a87d3 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:32.926907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-46b36055-c72e-4a85-ab26-0498b0277f99 None None] MFA Rules not processed for user `5a8c165029d547c184f642480d3e0f95`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:32.928207 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-46b36055-c72e-4a85-ab26-0498b0277f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:32.958880 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:32.965473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd99da-8bc8-4d25-9ce7-73ea6d52f789 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:32.966090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd99da-8bc8-4d25-9ce7-73ea6d52f789 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:32.966373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd99da-8bc8-4d25-9ce7-73ea6d52f789 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:32.966702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd99da-8bc8-4d25-9ce7-73ea6d52f789 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:32.968568 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8edd99da-8bc8-4d25-9ce7-73ea6d52f789 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:33.001424 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1117/2259] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:33.007468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b36055-c72e-4a85-ab26-0498b0277f99 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:33.047508 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:33.049921 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd99da-8bc8-4d25-9ce7-73ea6d52f789 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:33.051414 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd99da-8bc8-4d25-9ce7-73ea6d52f789 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:33.061407 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1143/2260] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:33.078105 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b36055-c72e-4a85-ab26-0498b0277f99 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:33.082809 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586424a-7dbe-4757-a711-2f5414207856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:33.083443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586424a-7dbe-4757-a711-2f5414207856 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:33.083721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586424a-7dbe-4757-a711-2f5414207856 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:33.083993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586424a-7dbe-4757-a711-2f5414207856 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:33.084680 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586424a-7dbe-4757-a711-2f5414207856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:33.087223 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586424a-7dbe-4757-a711-2f5414207856 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:33.183564 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1118/2261] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:33 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:33.184941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd99da-8bc8-4d25-9ce7-73ea6d52f789 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:33.197652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd99da-8bc8-4d25-9ce7-73ea6d52f789 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:33.259470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf22ede-4b41-433f-846b-ec5a8c7003a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:33.260078 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf22ede-4b41-433f-846b-ec5a8c7003a0 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:33.260356 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf22ede-4b41-433f-846b-ec5a8c7003a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:33.260627 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf22ede-4b41-433f-846b-ec5a8c7003a0 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:33.261262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf22ede-4b41-433f-846b-ec5a8c7003a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:33.263199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf22ede-4b41-433f-846b-ec5a8c7003a0 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:33.356292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1144/2262] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:33 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:33.360428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586424a-7dbe-4757-a711-2f5414207856 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:33.371943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586424a-7dbe-4757-a711-2f5414207856 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:33.376561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcb254-8265-45af-9f16-2175af7edd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:33.377159 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcb254-8265-45af-9f16-2175af7edd96 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:33.384691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcb254-8265-45af-9f16-2175af7edd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:33.384691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcb254-8265-45af-9f16-2175af7edd96 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:33.384691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dcb254-8265-45af-9f16-2175af7edd96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:33.384691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dcb254-8265-45af-9f16-2175af7edd96 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:33.390346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1119/2263] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:33.394550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf22ede-4b41-433f-846b-ec5a8c7003a0 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:33.404497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf22ede-4b41-433f-846b-ec5a8c7003a0 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:33.409359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd798f-1192-4625-9c76-8e8f8e1adff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:33.409982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd798f-1192-4625-9c76-8e8f8e1adff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:33.410254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd798f-1192-4625-9c76-8e8f8e1adff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:33.410522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd798f-1192-4625-9c76-8e8f8e1adff8 admin admin] PATH_INFO: `/v3/projects/39799ba20e4142c39ff54c78cfe9acf2/users/71f918a27879480ebcc526978035a82e/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:33.425133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd798f-1192-4625-9c76-8e8f8e1adff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39799ba20e4142c39ff54c78cfe9acf2, user_id=71f918a27879480ebcc526978035a82e, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:33.427892 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd798f-1192-4625-9c76-8e8f8e1adff8 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:33.516563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1145/2264] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:33 2022] PUT /identity/v3/projects/39799ba20e4142c39ff54c78cfe9acf2/users/71f918a27879480ebcc526978035a82e/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:33.521108 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcb254-8265-45af-9f16-2175af7edd96 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:33.531362 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcb254-8265-45af-9f16-2175af7edd96 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:33.590866 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9343c8-522c-4a66-a2fb-ad0d8b043f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:33.592079 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9343c8-522c-4a66-a2fb-ad0d8b043f79 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:33.592357 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9343c8-522c-4a66-a2fb-ad0d8b043f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:33.592749 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9343c8-522c-4a66-a2fb-ad0d8b043f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:33.593605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9343c8-522c-4a66-a2fb-ad0d8b043f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:33.595297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9343c8-522c-4a66-a2fb-ad0d8b043f79 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:33.606143 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1120/2265] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:33.610366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd798f-1192-4625-9c76-8e8f8e1adff8 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:33.622402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd798f-1192-4625-9c76-8e8f8e1adff8 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:33.627262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118df1b-1ead-4dbf-9cd9-eee3ea3f39fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:33.627963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118df1b-1ead-4dbf-9cd9-eee3ea3f39fc admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:33.628245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118df1b-1ead-4dbf-9cd9-eee3ea3f39fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:33.628516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118df1b-1ead-4dbf-9cd9-eee3ea3f39fc admin admin] PATH_INFO: `/v3/projects/39799ba20e4142c39ff54c78cfe9acf2/users/71f918a27879480ebcc526978035a82e/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:33.630659 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118df1b-1ead-4dbf-9cd9-eee3ea3f39fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=39799ba20e4142c39ff54c78cfe9acf2, user_id=71f918a27879480ebcc526978035a82e, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:33.633392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118df1b-1ead-4dbf-9cd9-eee3ea3f39fc admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:33.644495 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1146/2266] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:33 2022] PUT /identity/v3/projects/39799ba20e4142c39ff54c78cfe9acf2/users/71f918a27879480ebcc526978035a82e/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:33.656517 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a938e-4c92-4495-af5d-412c28810700 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:33.656933 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a938e-4c92-4495-af5d-412c28810700 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:33.657223 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a938e-4c92-4495-af5d-412c28810700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:33.687639 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-385a938e-4c92-4495-af5d-412c28810700 None None] MFA Rules not processed for user `71f918a27879480ebcc526978035a82e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:33.688865 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-385a938e-4c92-4495-af5d-412c28810700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:33.721264 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:33.766615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1121/2267] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:33.778339 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118df1b-1ead-4dbf-9cd9-eee3ea3f39fc admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:33.839475 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118df1b-1ead-4dbf-9cd9-eee3ea3f39fc admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:33.843051 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966712b-8e01-453b-a199-e85d027bce41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:33.843680 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966712b-8e01-453b-a199-e85d027bce41 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:33.843957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966712b-8e01-453b-a199-e85d027bce41 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:33.844225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966712b-8e01-453b-a199-e85d027bce41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:33.907478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966712b-8e01-453b-a199-e85d027bce41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:33.910231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966712b-8e01-453b-a199-e85d027bce41 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:33.918856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1147/2268] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:35.881618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a938e-4c92-4495-af5d-412c28810700 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:35.889181 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-385a938e-4c92-4495-af5d-412c28810700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:35.906759 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966712b-8e01-453b-a199-e85d027bce41 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:35.908223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-3966712b-8e01-453b-a199-e85d027bce41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:35.915745 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:35.923299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966712b-8e01-453b-a199-e85d027bce41 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:35.925063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6130b-e4e0-48f8-aaa8-d9caa99817bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:35.925662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6130b-e4e0-48f8-aaa8-d9caa99817bd service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:35.925957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6130b-e4e0-48f8-aaa8-d9caa99817bd service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:35.926225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6130b-e4e0-48f8-aaa8-d9caa99817bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:35.955198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a938e-4c92-4495-af5d-412c28810700 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:35.961068 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebd7c2-551a-479d-8543-a1c6bb368f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:35.961845 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebd7c2-551a-479d-8543-a1c6bb368f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:35.962127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebd7c2-551a-479d-8543-a1c6bb368f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:35.962438 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebd7c2-551a-479d-8543-a1c6bb368f41 admin admin] PATH_INFO: `/v3/users/e8579725af034affb4f54b532b1f9313` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:35.964208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ebd7c2-551a-479d-8543-a1c6bb368f41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8579725af034affb4f54b532b1f9313)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:35.965645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ebd7c2-551a-479d-8543-a1c6bb368f41 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:35.997522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6130b-e4e0-48f8-aaa8-d9caa99817bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:35.998972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6130b-e4e0-48f8-aaa8-d9caa99817bd service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:36.008721 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1148/2269] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:36.191955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a1ebd7c2-551a-479d-8543-a1c6bb368f41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8579725af034affb4f54b532b1f9313'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.199644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a1ebd7c2-551a-479d-8543-a1c6bb368f41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8579725af034affb4f54b532b1f9313'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.200389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a1ebd7c2-551a-479d-8543-a1c6bb368f41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8579725af034affb4f54b532b1f9313'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.286747 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a1ebd7c2-551a-479d-8543-a1c6bb368f41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8579725af034affb4f54b532b1f9313'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.299433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-a1ebd7c2-551a-479d-8543-a1c6bb368f41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8579725af034affb4f54b532b1f9313'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:36.305868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1122/2270] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:35 2022] DELETE /identity/v3/users/e8579725af034affb4f54b532b1f9313 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:36.310376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ae8dd-8b19-42d9-9977-bada1a2cdc9f None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:36.310659 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ae8dd-8b19-42d9-9977-bada1a2cdc9f None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:36.310927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ae8dd-8b19-42d9-9977-bada1a2cdc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:36.335683 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-cd9ae8dd-8b19-42d9-9977-bada1a2cdc9f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:36.336990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-cd9ae8dd-8b19-42d9-9977-bada1a2cdc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:36.394318 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1149/2271] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:36.404224 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebd7c2-551a-479d-8543-a1c6bb368f41 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:36.405785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a1ebd7c2-551a-479d-8543-a1c6bb368f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:36.452992 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:36.464244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebd7c2-551a-479d-8543-a1c6bb368f41 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:36.468082 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d35846-e77c-4a95-900e-567f7f9184fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:36.468719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d35846-e77c-4a95-900e-567f7f9184fb service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:36.468995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d35846-e77c-4a95-900e-567f7f9184fb service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:36.469261 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d35846-e77c-4a95-900e-567f7f9184fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:36.478322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d35846-e77c-4a95-900e-567f7f9184fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:36.480062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d35846-e77c-4a95-900e-567f7f9184fb service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:36.489306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1123/2272] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:36.969450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ae8dd-8b19-42d9-9977-bada1a2cdc9f None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:36.969450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-cd9ae8dd-8b19-42d9-9977-bada1a2cdc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:36.982140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ae8dd-8b19-42d9-9977-bada1a2cdc9f None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:36.987340 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:36.988227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:36.988639 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:36.988949 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e admin admin] PATH_INFO: `/v3/projects/453789ea3de34512885d8596d1aee6de` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:36.990063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=453789ea3de34512885d8596d1aee6de)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:36.992333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:37.156405 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '453789ea3de34512885d8596d1aee6de', 'request_id': 'req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:37.176375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '453789ea3de34512885d8596d1aee6de', 'request_id': 'req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:37.245756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '453789ea3de34512885d8596d1aee6de', 'request_id': 'req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:37.260727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e admin admin] The token cache is being invalidate because project 453789ea3de34512885d8596d1aee6de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:37.261015 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:37.261781 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:37.264270 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1150/2273] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:36 2022] DELETE /identity/v3/projects/453789ea3de34512885d8596d1aee6de => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:37.267979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d35846-e77c-4a95-900e-567f7f9184fb service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:37.287541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d35846-e77c-4a95-900e-567f7f9184fb service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:37.358564 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb2713-32ce-4f92-bbd4-3fb712cc816f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:37.359346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb2713-32ce-4f92-bbd4-3fb712cc816f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:37.359692 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb2713-32ce-4f92-bbd4-3fb712cc816f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:37.360041 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb2713-32ce-4f92-bbd4-3fb712cc816f admin admin] PATH_INFO: `/v3/users/6672d70199994c6882cbe8d6d4cc7857` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:37.361271 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb2713-32ce-4f92-bbd4-3fb712cc816f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6672d70199994c6882cbe8d6d4cc7857)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:37.363333 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb2713-32ce-4f92-bbd4-3fb712cc816f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:37.584171 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b9bb2713-32ce-4f92-bbd4-3fb712cc816f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6672d70199994c6882cbe8d6d4cc7857'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:37.591348 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b9bb2713-32ce-4f92-bbd4-3fb712cc816f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6672d70199994c6882cbe8d6d4cc7857'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:37.592062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b9bb2713-32ce-4f92-bbd4-3fb712cc816f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6672d70199994c6882cbe8d6d4cc7857'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:37.704324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b9bb2713-32ce-4f92-bbd4-3fb712cc816f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6672d70199994c6882cbe8d6d4cc7857'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:37.718325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b9bb2713-32ce-4f92-bbd4-3fb712cc816f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6672d70199994c6882cbe8d6d4cc7857'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:37.726202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1124/2274] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:37 2022] DELETE /identity/v3/users/6672d70199994c6882cbe8d6d4cc7857 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:37.882547 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:37.884004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:37.913918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:37.956229 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607f1bf-4520-4c85-b51a-3f4e3a43db6e admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:38.011232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a81e07-0e2f-482d-bd4c-a7e628328d0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:38.011900 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a81e07-0e2f-482d-bd4c-a7e628328d0a service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:38.012150 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a81e07-0e2f-482d-bd4c-a7e628328d0a service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:38.012426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a81e07-0e2f-482d-bd4c-a7e628328d0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:38.014934 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-05a81e07-0e2f-482d-bd4c-a7e628328d0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:38.097677 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a81e07-0e2f-482d-bd4c-a7e628328d0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:38.099478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a81e07-0e2f-482d-bd4c-a7e628328d0a service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:38.112324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1151/2275] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:38.266963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb2713-32ce-4f92-bbd4-3fb712cc816f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:38.267732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b9bb2713-32ce-4f92-bbd4-3fb712cc816f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:38.347033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb2713-32ce-4f92-bbd4-3fb712cc816f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:38.352451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:38.353084 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:38.353375 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:38.353820 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0 admin admin] PATH_INFO: `/v3/projects/1bf364aeda6e4a57abf71b1828b16fcb` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:38.355312 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bf364aeda6e4a57abf71b1828b16fcb)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:38.358383 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:38.369522 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:38.516196 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bf364aeda6e4a57abf71b1828b16fcb', 'request_id': 'req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:38.523408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bf364aeda6e4a57abf71b1828b16fcb', 'request_id': 'req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:38.531981 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bf364aeda6e4a57abf71b1828b16fcb', 'request_id': 'req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:38.646426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0 admin admin] The token cache is being invalidate because project 1bf364aeda6e4a57abf71b1828b16fcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:38.646720 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:38.647403 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:38.652137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1125/2276] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:38 2022] DELETE /identity/v3/projects/1bf364aeda6e4a57abf71b1828b16fcb => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:38.653165 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a81e07-0e2f-482d-bd4c-a7e628328d0a service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:38.663953 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a81e07-0e2f-482d-bd4c-a7e628328d0a service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:38.722063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8a2c2-6b84-4efe-a05f-3fac7fff25b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:38.722932 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8a2c2-6b84-4efe-a05f-3fac7fff25b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:38.723392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8a2c2-6b84-4efe-a05f-3fac7fff25b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:38.723809 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8a2c2-6b84-4efe-a05f-3fac7fff25b5 admin admin] PATH_INFO: `/v3/users/c971cafede0f4454a3fa53998a1d310b` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:38.726144 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8a2c2-6b84-4efe-a05f-3fac7fff25b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c971cafede0f4454a3fa53998a1d310b)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:38.728516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8a2c2-6b84-4efe-a05f-3fac7fff25b5 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:39.020628 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-34f8a2c2-6b84-4efe-a05f-3fac7fff25b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c971cafede0f4454a3fa53998a1d310b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:39.027559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-34f8a2c2-6b84-4efe-a05f-3fac7fff25b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c971cafede0f4454a3fa53998a1d310b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:39.039959 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-34f8a2c2-6b84-4efe-a05f-3fac7fff25b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c971cafede0f4454a3fa53998a1d310b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:39.040678 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-34f8a2c2-6b84-4efe-a05f-3fac7fff25b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c971cafede0f4454a3fa53998a1d310b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:39.041416 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-34f8a2c2-6b84-4efe-a05f-3fac7fff25b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c971cafede0f4454a3fa53998a1d310b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:39.162180 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1152/2277] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:38 2022] DELETE /identity/v3/users/c971cafede0f4454a3fa53998a1d310b => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:39.287294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:39.632751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:39.634281 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:39.657453 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252cae3-49f9-4b4b-8ea3-6dfcfc0a0bc0 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:39.719825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4d59f-2916-4a38-b211-16dd0abbfa8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:39.720562 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4d59f-2916-4a38-b211-16dd0abbfa8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:39.720854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4d59f-2916-4a38-b211-16dd0abbfa8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:39.721230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4d59f-2916-4a38-b211-16dd0abbfa8a admin admin] PATH_INFO: `/v3/projects/d86302db7f1e4f418dff4ab9223cd920` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:39.722387 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4d59f-2916-4a38-b211-16dd0abbfa8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d86302db7f1e4f418dff4ab9223cd920)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:39.724333 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4d59f-2916-4a38-b211-16dd0abbfa8a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:39.813125 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3fc4d59f-2916-4a38-b211-16dd0abbfa8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd86302db7f1e4f418dff4ab9223cd920', 'request_id': 'req-3fc4d59f-2916-4a38-b211-16dd0abbfa8a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:39.819938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3fc4d59f-2916-4a38-b211-16dd0abbfa8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd86302db7f1e4f418dff4ab9223cd920', 'request_id': 'req-3fc4d59f-2916-4a38-b211-16dd0abbfa8a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:39.825655 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3fc4d59f-2916-4a38-b211-16dd0abbfa8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd86302db7f1e4f418dff4ab9223cd920', 'request_id': 'req-3fc4d59f-2916-4a38-b211-16dd0abbfa8a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:39.967000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3fc4d59f-2916-4a38-b211-16dd0abbfa8a admin admin] The token cache is being invalidate because project d86302db7f1e4f418dff4ab9223cd920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:39.967314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3fc4d59f-2916-4a38-b211-16dd0abbfa8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:39.967940 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3fc4d59f-2916-4a38-b211-16dd0abbfa8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:39.969418 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1126/2278] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:39 2022] DELETE /identity/v3/projects/d86302db7f1e4f418dff4ab9223cd920 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:39.974308 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d599c-4a75-4d84-9604-336856afadce None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:39.974594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d599c-4a75-4d84-9604-336856afadce None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:39.974870 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d599c-4a75-4d84-9604-336856afadce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:39.995186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-370d599c-4a75-4d84-9604-336856afadce None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:39.996609 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-370d599c-4a75-4d84-9604-336856afadce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:40.064167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1153/2279] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:40.072071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f27fa-f3c0-42b0-a20a-e569c87d0ffd None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:40.072360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f27fa-f3c0-42b0-a20a-e569c87d0ffd None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:40.072634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f27fa-f3c0-42b0-a20a-e569c87d0ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:40.097310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f32f27fa-f3c0-42b0-a20a-e569c87d0ffd None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:40.098636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f32f27fa-f3c0-42b0-a20a-e569c87d0ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:40.108798 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1127/2280] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:40.113211 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d599c-4a75-4d84-9604-336856afadce None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:40.123418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d599c-4a75-4d84-9604-336856afadce None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:40.129407 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fad7f7-33ca-4d49-93df-bc2c61bf625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:40.130150 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fad7f7-33ca-4d49-93df-bc2c61bf625e admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:40.130330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fad7f7-33ca-4d49-93df-bc2c61bf625e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:40.130597 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fad7f7-33ca-4d49-93df-bc2c61bf625e admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:40.131268 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fad7f7-33ca-4d49-93df-bc2c61bf625e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:40.131615 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fad7f7-33ca-4d49-93df-bc2c61bf625e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:40.133500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fad7f7-33ca-4d49-93df-bc2c61bf625e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:40.146224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1154/2281] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:58:40.150866 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f27fa-f3c0-42b0-a20a-e569c87d0ffd None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:40.160419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f27fa-f3c0-42b0-a20a-e569c87d0ffd None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:40.166110 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920a644-eb6d-4284-8831-7f6800028b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:40.166727 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920a644-eb6d-4284-8831-7f6800028b3e admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:40.167006 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920a644-eb6d-4284-8831-7f6800028b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:40.167309 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920a644-eb6d-4284-8831-7f6800028b3e admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:40.167955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920a644-eb6d-4284-8831-7f6800028b3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:40.170025 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920a644-eb6d-4284-8831-7f6800028b3e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:40.265398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:40.274471 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1128/2282] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:40 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:40.278533 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fad7f7-33ca-4d49-93df-bc2c61bf625e admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:40.305681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fad7f7-33ca-4d49-93df-bc2c61bf625e admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:40.366980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb21be-869c-4e1f-9487-450422a77145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:40.367615 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb21be-869c-4e1f-9487-450422a77145 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:40.367890 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb21be-869c-4e1f-9487-450422a77145 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:40.368160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb21be-869c-4e1f-9487-450422a77145 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:40.368784 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb21be-869c-4e1f-9487-450422a77145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:40.371390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb21be-869c-4e1f-9487-450422a77145 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:40.492906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1155/2283] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:40 2022] POST /identity/v3/users => generated 284 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:40.497127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920a644-eb6d-4284-8831-7f6800028b3e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:40.507757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920a644-eb6d-4284-8831-7f6800028b3e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:40.512225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081ca5d-29dc-4009-b4ad-216e5d659b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:40.512813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081ca5d-29dc-4009-b4ad-216e5d659b23 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:40.513091 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081ca5d-29dc-4009-b4ad-216e5d659b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:40.513361 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081ca5d-29dc-4009-b4ad-216e5d659b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:40.514071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081ca5d-29dc-4009-b4ad-216e5d659b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:40.515707 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081ca5d-29dc-4009-b4ad-216e5d659b23 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:40.531507 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1129/2284] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:40.534208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb21be-869c-4e1f-9487-450422a77145 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:40.544039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb21be-869c-4e1f-9487-450422a77145 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:40.548534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8759a-f4e6-4287-8e6e-8b50bee21d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:40.549136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8759a-f4e6-4287-8e6e-8b50bee21d50 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:40.549385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8759a-f4e6-4287-8e6e-8b50bee21d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:40.549649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8759a-f4e6-4287-8e6e-8b50bee21d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:40.550278 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8759a-f4e6-4287-8e6e-8b50bee21d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:40.551891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8759a-f4e6-4287-8e6e-8b50bee21d50 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:40.561851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1156/2285] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:58:40.569614 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081ca5d-29dc-4009-b4ad-216e5d659b23 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:40.579481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081ca5d-29dc-4009-b4ad-216e5d659b23 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:40.584320 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb8214-7e49-41ec-9fad-3c5cc3d0c9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:40.585580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb8214-7e49-41ec-9fad-3c5cc3d0c9d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:40.585859 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb8214-7e49-41ec-9fad-3c5cc3d0c9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:40.588085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb8214-7e49-41ec-9fad-3c5cc3d0c9d4 admin admin] PATH_INFO: `/v3/projects/3bd055911f224d00af5f07debb5a6d46/users/664c4d177c8349b6825b9f7fb738c5af/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:40.601054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb8214-7e49-41ec-9fad-3c5cc3d0c9d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd055911f224d00af5f07debb5a6d46, user_id=664c4d177c8349b6825b9f7fb738c5af, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:40.603752 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb8214-7e49-41ec-9fad-3c5cc3d0c9d4 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:40.753981 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1130/2286] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:40 2022] PUT /identity/v3/projects/3bd055911f224d00af5f07debb5a6d46/users/664c4d177c8349b6825b9f7fb738c5af/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:40.765011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80137f5c-42ff-45c0-9156-6a016e5f437a None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:40.765325 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80137f5c-42ff-45c0-9156-6a016e5f437a None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:40.765726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80137f5c-42ff-45c0-9156-6a016e5f437a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:40.804124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-80137f5c-42ff-45c0-9156-6a016e5f437a None None] MFA Rules not processed for user `664c4d177c8349b6825b9f7fb738c5af`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:40.805664 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-80137f5c-42ff-45c0-9156-6a016e5f437a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:40.810016 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:40.893236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1157/2287] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:40.905945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb8214-7e49-41ec-9fad-3c5cc3d0c9d4 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:40.907953 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b4fb8214-7e49-41ec-9fad-3c5cc3d0c9d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:40.968661 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb8214-7e49-41ec-9fad-3c5cc3d0c9d4 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:40.972395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85660b-6512-4765-b9dd-22049a40a7a8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfda7307e703427faaf599e32b91bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:40.972996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85660b-6512-4765-b9dd-22049a40a7a8 service glance] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:40.973273 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85660b-6512-4765-b9dd-22049a40a7a8 service glance] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:40.973541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85660b-6512-4765-b9dd-22049a40a7a8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:40.981790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85660b-6512-4765-b9dd-22049a40a7a8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:40.983235 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85660b-6512-4765-b9dd-22049a40a7a8 service glance] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:40.992485 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1131/2288] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:43.468466 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80137f5c-42ff-45c0-9156-6a016e5f437a None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:43.469914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-80137f5c-42ff-45c0-9156-6a016e5f437a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:43.540248 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80137f5c-42ff-45c0-9156-6a016e5f437a None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:43.544991 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457156a4-99cb-4b99-b0c9-b93a3528727e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:43.546811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457156a4-99cb-4b99-b0c9-b93a3528727e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:43.547315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457156a4-99cb-4b99-b0c9-b93a3528727e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:43.547589 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457156a4-99cb-4b99-b0c9-b93a3528727e admin admin] PATH_INFO: `/v3/users/ed6ea1fff22445468245117de885c0eb` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:43.548810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457156a4-99cb-4b99-b0c9-b93a3528727e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed6ea1fff22445468245117de885c0eb)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:43.550738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457156a4-99cb-4b99-b0c9-b93a3528727e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:43.769063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-457156a4-99cb-4b99-b0c9-b93a3528727e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed6ea1fff22445468245117de885c0eb'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:43.777779 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-457156a4-99cb-4b99-b0c9-b93a3528727e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed6ea1fff22445468245117de885c0eb'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:43.790527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-457156a4-99cb-4b99-b0c9-b93a3528727e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed6ea1fff22445468245117de885c0eb'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:43.791232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-457156a4-99cb-4b99-b0c9-b93a3528727e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed6ea1fff22445468245117de885c0eb'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:43.791843 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-457156a4-99cb-4b99-b0c9-b93a3528727e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed6ea1fff22445468245117de885c0eb'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:43.863271 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1158/2289] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:43 2022] DELETE /identity/v3/users/ed6ea1fff22445468245117de885c0eb => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:43.868265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aefaa9-2ff5-4dfe-846f-6a507c127986 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:43.868746 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aefaa9-2ff5-4dfe-846f-6a507c127986 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:43.868870 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aefaa9-2ff5-4dfe-846f-6a507c127986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:43.888011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-46aefaa9-2ff5-4dfe-846f-6a507c127986 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:43.889206 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-46aefaa9-2ff5-4dfe-846f-6a507c127986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:43.954503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1132/2290] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:43.964832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457156a4-99cb-4b99-b0c9-b93a3528727e admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:43.966652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-457156a4-99cb-4b99-b0c9-b93a3528727e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:43.998048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:44.033002 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457156a4-99cb-4b99-b0c9-b93a3528727e admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:44.037333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff72e83-4ce5-4059-b9f5-73a0de140a23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:44.038705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff72e83-4ce5-4059-b9f5-73a0de140a23 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:44.038975 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff72e83-4ce5-4059-b9f5-73a0de140a23 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:44.039279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff72e83-4ce5-4059-b9f5-73a0de140a23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:44.050777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff72e83-4ce5-4059-b9f5-73a0de140a23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:44.052186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff72e83-4ce5-4059-b9f5-73a0de140a23 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:44.062101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1159/2291] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:44.654704 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aefaa9-2ff5-4dfe-846f-6a507c127986 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:44.656688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-46aefaa9-2ff5-4dfe-846f-6a507c127986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:44.691394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aefaa9-2ff5-4dfe-846f-6a507c127986 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:44.699451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffced0-eca8-4944-b720-dcd7928df4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:44.700755 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffced0-eca8-4944-b720-dcd7928df4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:44.700755 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffced0-eca8-4944-b720-dcd7928df4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:44.700998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffced0-eca8-4944-b720-dcd7928df4ca admin admin] PATH_INFO: `/v3/projects/aa8c3314237d46ff89a34b0064ad2714` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:44.702112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffced0-eca8-4944-b720-dcd7928df4ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa8c3314237d46ff89a34b0064ad2714)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:44.704103 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffced0-eca8-4944-b720-dcd7928df4ca admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:44.839562 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-deffced0-eca8-4944-b720-dcd7928df4ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa8c3314237d46ff89a34b0064ad2714', 'request_id': 'req-deffced0-eca8-4944-b720-dcd7928df4ca'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:44.845983 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-deffced0-eca8-4944-b720-dcd7928df4ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa8c3314237d46ff89a34b0064ad2714', 'request_id': 'req-deffced0-eca8-4944-b720-dcd7928df4ca'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:44.851932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:44.853377 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-deffced0-eca8-4944-b720-dcd7928df4ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa8c3314237d46ff89a34b0064ad2714', 'request_id': 'req-deffced0-eca8-4944-b720-dcd7928df4ca'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:44.972798 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-deffced0-eca8-4944-b720-dcd7928df4ca admin admin] The token cache is being invalidate because project aa8c3314237d46ff89a34b0064ad2714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:44.973449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-deffced0-eca8-4944-b720-dcd7928df4ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:44.973834 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-deffced0-eca8-4944-b720-dcd7928df4ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:44.976039 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1133/2292] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:44 2022] DELETE /identity/v3/projects/aa8c3314237d46ff89a34b0064ad2714 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:44.980270 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff72e83-4ce5-4059-b9f5-73a0de140a23 service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:44.992434 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff72e83-4ce5-4059-b9f5-73a0de140a23 service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:45.053702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537eed0-f373-4b54-836e-8492f5941981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:45.054342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537eed0-f373-4b54-836e-8492f5941981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:45.054628 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537eed0-f373-4b54-836e-8492f5941981 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:45.054926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537eed0-f373-4b54-836e-8492f5941981 admin admin] PATH_INFO: `/v3/users/5a8c165029d547c184f642480d3e0f95` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:45.056101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537eed0-f373-4b54-836e-8492f5941981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a8c165029d547c184f642480d3e0f95)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:45.058085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537eed0-f373-4b54-836e-8492f5941981 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:45.257825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a537eed0-f373-4b54-836e-8492f5941981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a8c165029d547c184f642480d3e0f95'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:45.271496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a537eed0-f373-4b54-836e-8492f5941981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a8c165029d547c184f642480d3e0f95'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:45.284320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a537eed0-f373-4b54-836e-8492f5941981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a8c165029d547c184f642480d3e0f95'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:45.285052 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a537eed0-f373-4b54-836e-8492f5941981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a8c165029d547c184f642480d3e0f95'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:45.285652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a537eed0-f373-4b54-836e-8492f5941981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a8c165029d547c184f642480d3e0f95'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:45.377481 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1160/2293] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:44 2022] DELETE /identity/v3/users/5a8c165029d547c184f642480d3e0f95 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:45.911936 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffced0-eca8-4944-b720-dcd7928df4ca admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:45.914382 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-deffced0-eca8-4944-b720-dcd7928df4ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:45.934021 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffced0-eca8-4944-b720-dcd7928df4ca admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:46.004246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79e2b5-aec1-4148-98f9-7c65d4450cc1 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:46.004246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79e2b5-aec1-4148-98f9-7c65d4450cc1 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:46.004246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79e2b5-aec1-4148-98f9-7c65d4450cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:46.015489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c8cee-0186-426b-b39c-e323f65e3975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:46.016124 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c8cee-0186-426b-b39c-e323f65e3975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:46.016402 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c8cee-0186-426b-b39c-e323f65e3975 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:46.016708 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c8cee-0186-426b-b39c-e323f65e3975 admin admin] PATH_INFO: `/v3/projects/00ec3dc042e645bd823869970c51903e` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:46.017868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c8cee-0186-426b-b39c-e323f65e3975 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00ec3dc042e645bd823869970c51903e)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:46.019881 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c8cee-0186-426b-b39c-e323f65e3975 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:46.020871 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-eb79e2b5-aec1-4148-98f9-7c65d4450cc1 None None] MFA Rules not processed for user `39d98294ba884280b4971af36a9fbd6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:46.029339 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-eb79e2b5-aec1-4148-98f9-7c65d4450cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:46.080121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:46.094211 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1161/2294] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:46.159849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ae0c8cee-0186-426b-b39c-e323f65e3975 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00ec3dc042e645bd823869970c51903e', 'request_id': 'req-ae0c8cee-0186-426b-b39c-e323f65e3975'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.167060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ae0c8cee-0186-426b-b39c-e323f65e3975 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00ec3dc042e645bd823869970c51903e', 'request_id': 'req-ae0c8cee-0186-426b-b39c-e323f65e3975'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.182507 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ae0c8cee-0186-426b-b39c-e323f65e3975 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00ec3dc042e645bd823869970c51903e', 'request_id': 'req-ae0c8cee-0186-426b-b39c-e323f65e3975'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.345197 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ae0c8cee-0186-426b-b39c-e323f65e3975 admin admin] The token cache is being invalidate because project 00ec3dc042e645bd823869970c51903e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:46.345520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ae0c8cee-0186-426b-b39c-e323f65e3975 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.346143 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ae0c8cee-0186-426b-b39c-e323f65e3975 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.347675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1134/2295] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:45 2022] DELETE /identity/v3/projects/00ec3dc042e645bd823869970c51903e => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:46.352108 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79e2b5-aec1-4148-98f9-7c65d4450cc1 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:46.353622 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-eb79e2b5-aec1-4148-98f9-7c65d4450cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:46.367726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79e2b5-aec1-4148-98f9-7c65d4450cc1 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:46.426857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95a8a7-0b2a-4e60-8d48-1a489bec461a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:46.427659 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95a8a7-0b2a-4e60-8d48-1a489bec461a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:46.427961 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95a8a7-0b2a-4e60-8d48-1a489bec461a admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:46.428826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95a8a7-0b2a-4e60-8d48-1a489bec461a admin admin] PATH_INFO: `/v3/users/71f918a27879480ebcc526978035a82e` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:46.429954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95a8a7-0b2a-4e60-8d48-1a489bec461a admin admin] RBAC: Authorizing `identity:delete_user(user_id=71f918a27879480ebcc526978035a82e)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:46.431906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95a8a7-0b2a-4e60-8d48-1a489bec461a admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:46.666069 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-fe95a8a7-0b2a-4e60-8d48-1a489bec461a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71f918a27879480ebcc526978035a82e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.672950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-fe95a8a7-0b2a-4e60-8d48-1a489bec461a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71f918a27879480ebcc526978035a82e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.685852 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-fe95a8a7-0b2a-4e60-8d48-1a489bec461a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71f918a27879480ebcc526978035a82e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.686559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-fe95a8a7-0b2a-4e60-8d48-1a489bec461a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71f918a27879480ebcc526978035a82e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.687232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-fe95a8a7-0b2a-4e60-8d48-1a489bec461a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71f918a27879480ebcc526978035a82e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:46.798550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1162/2296] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:46 2022] DELETE /identity/v3/users/71f918a27879480ebcc526978035a82e => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:47.342406 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c8cee-0186-426b-b39c-e323f65e3975 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:47.343899 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ae0c8cee-0186-426b-b39c-e323f65e3975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:47.360933 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c8cee-0186-426b-b39c-e323f65e3975 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:47.442689 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43107dd9-4caf-44d9-a4d9-998896dfbef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:47.444001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43107dd9-4caf-44d9-a4d9-998896dfbef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:47.444303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43107dd9-4caf-44d9-a4d9-998896dfbef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:47.444794 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43107dd9-4caf-44d9-a4d9-998896dfbef2 admin admin] PATH_INFO: `/v3/projects/39799ba20e4142c39ff54c78cfe9acf2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:47.446414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43107dd9-4caf-44d9-a4d9-998896dfbef2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39799ba20e4142c39ff54c78cfe9acf2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:47.448474 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43107dd9-4caf-44d9-a4d9-998896dfbef2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:47.474607 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:47.590165 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-43107dd9-4caf-44d9-a4d9-998896dfbef2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39799ba20e4142c39ff54c78cfe9acf2', 'request_id': 'req-43107dd9-4caf-44d9-a4d9-998896dfbef2'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.596448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-43107dd9-4caf-44d9-a4d9-998896dfbef2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39799ba20e4142c39ff54c78cfe9acf2', 'request_id': 'req-43107dd9-4caf-44d9-a4d9-998896dfbef2'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.602038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-43107dd9-4caf-44d9-a4d9-998896dfbef2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39799ba20e4142c39ff54c78cfe9acf2', 'request_id': 'req-43107dd9-4caf-44d9-a4d9-998896dfbef2'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.691190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-43107dd9-4caf-44d9-a4d9-998896dfbef2 admin admin] The token cache is being invalidate because project 39799ba20e4142c39ff54c78cfe9acf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:47.691566 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-43107dd9-4caf-44d9-a4d9-998896dfbef2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.692314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-43107dd9-4caf-44d9-a4d9-998896dfbef2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:47.695256 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1135/2297] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:47 2022] DELETE /identity/v3/projects/39799ba20e4142c39ff54c78cfe9acf2 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:47.702058 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece7064-60c6-499f-a643-f5dea296fbe4 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:47.702581 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece7064-60c6-499f-a643-f5dea296fbe4 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:47.702581 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece7064-60c6-499f-a643-f5dea296fbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:47.724924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-bece7064-60c6-499f-a643-f5dea296fbe4 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:47.726162 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-bece7064-60c6-499f-a643-f5dea296fbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:47.788229 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1163/2298] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:47.796242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed46229-f7c9-4bee-bef9-68170c53d96c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:47.796527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed46229-f7c9-4bee-bef9-68170c53d96c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:47.796800 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed46229-f7c9-4bee-bef9-68170c53d96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:47.817998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1ed46229-f7c9-4bee-bef9-68170c53d96c None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:47.819766 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1ed46229-f7c9-4bee-bef9-68170c53d96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:47.835064 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1136/2299] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:47.837039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece7064-60c6-499f-a643-f5dea296fbe4 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:47.847456 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece7064-60c6-499f-a643-f5dea296fbe4 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:47.852713 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd29d8c-5134-4b4c-97f5-d2906550f9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:47.853422 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd29d8c-5134-4b4c-97f5-d2906550f9d1 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:47.854057 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd29d8c-5134-4b4c-97f5-d2906550f9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:47.854416 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd29d8c-5134-4b4c-97f5-d2906550f9d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:47.855225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd29d8c-5134-4b4c-97f5-d2906550f9d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:47.855589 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd29d8c-5134-4b4c-97f5-d2906550f9d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:47.857227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd29d8c-5134-4b4c-97f5-d2906550f9d1 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:47.864863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:47.870711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1164/2300] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:58:47.875268 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed46229-f7c9-4bee-bef9-68170c53d96c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:47.889391 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed46229-f7c9-4bee-bef9-68170c53d96c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:47.895198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b20d81-44d5-4123-9b85-baccf2087d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:47.895198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b20d81-44d5-4123-9b85-baccf2087d90 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:47.895198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b20d81-44d5-4123-9b85-baccf2087d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:47.895488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b20d81-44d5-4123-9b85-baccf2087d90 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:47.896175 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b20d81-44d5-4123-9b85-baccf2087d90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:47.898086 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b20d81-44d5-4123-9b85-baccf2087d90 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.009639 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1137/2301] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:47 2022] POST /identity/v3/projects => generated 371 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:48.013635 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd29d8c-5134-4b4c-97f5-d2906550f9d1 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:48.024774 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd29d8c-5134-4b4c-97f5-d2906550f9d1 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.106811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a25bb-effe-4dc5-9e95-6e69d2918177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.107454 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a25bb-effe-4dc5-9e95-6e69d2918177 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.107733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a25bb-effe-4dc5-9e95-6e69d2918177 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.108007 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a25bb-effe-4dc5-9e95-6e69d2918177 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.108654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a25bb-effe-4dc5-9e95-6e69d2918177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.110535 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a25bb-effe-4dc5-9e95-6e69d2918177 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.236144 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1165/2302] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:48 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:48.240223 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b20d81-44d5-4123-9b85-baccf2087d90 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:48.251748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b20d81-44d5-4123-9b85-baccf2087d90 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.257018 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33552390-349a-41ce-9d49-b6bb0018dbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.257616 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33552390-349a-41ce-9d49-b6bb0018dbba admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.257896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33552390-349a-41ce-9d49-b6bb0018dbba admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.258165 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33552390-349a-41ce-9d49-b6bb0018dbba admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.258814 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33552390-349a-41ce-9d49-b6bb0018dbba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.260500 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33552390-349a-41ce-9d49-b6bb0018dbba admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.270879 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1138/2303] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:48.275593 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a25bb-effe-4dc5-9e95-6e69d2918177 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:48.286045 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a25bb-effe-4dc5-9e95-6e69d2918177 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.293471 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febbaf6-5480-4acd-b286-6cfc6192bda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.294089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febbaf6-5480-4acd-b286-6cfc6192bda4 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.294366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febbaf6-5480-4acd-b286-6cfc6192bda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.294636 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febbaf6-5480-4acd-b286-6cfc6192bda4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.295314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7febbaf6-5480-4acd-b286-6cfc6192bda4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.296941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7febbaf6-5480-4acd-b286-6cfc6192bda4 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.307650 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1166/2304] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:48.311931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33552390-349a-41ce-9d49-b6bb0018dbba admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:48.323594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33552390-349a-41ce-9d49-b6bb0018dbba admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.329187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3352011-449e-49d4-93cb-f73100c21e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.329921 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3352011-449e-49d4-93cb-f73100c21e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.330262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3352011-449e-49d4-93cb-f73100c21e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.330653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3352011-449e-49d4-93cb-f73100c21e05 admin admin] PATH_INFO: `/v3/projects/1c701547b70343aea7503b7d7b05ecb7/users/81b90c27ef274539895af6a0918cc0dd/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.352001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3352011-449e-49d4-93cb-f73100c21e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c701547b70343aea7503b7d7b05ecb7, user_id=81b90c27ef274539895af6a0918cc0dd, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.368423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3352011-449e-49d4-93cb-f73100c21e05 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.453186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1139/2305] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:48 2022] PUT /identity/v3/projects/1c701547b70343aea7503b7d7b05ecb7/users/81b90c27ef274539895af6a0918cc0dd/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:48.458878 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96c108-c879-4b80-b052-23ad94e52e66 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.459926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96c108-c879-4b80-b052-23ad94e52e66 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.460828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96c108-c879-4b80-b052-23ad94e52e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.524819 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-dd96c108-c879-4b80-b052-23ad94e52e66 None None] MFA Rules not processed for user `81b90c27ef274539895af6a0918cc0dd`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:48.526218 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-dd96c108-c879-4b80-b052-23ad94e52e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:48.619822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1167/2306] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:48.631458 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3352011-449e-49d4-93cb-f73100c21e05 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:48.633439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a3352011-449e-49d4-93cb-f73100c21e05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:48.700192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3352011-449e-49d4-93cb-f73100c21e05 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:48.704367 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cd5e8-a8c9-4b7a-9199-baab23599e0b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfda7307e703427faaf599e32b91bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:48.704943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cd5e8-a8c9-4b7a-9199-baab23599e0b service glance] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:48.705219 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cd5e8-a8c9-4b7a-9199-baab23599e0b service glance] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:48.705486 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cd5e8-a8c9-4b7a-9199-baab23599e0b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:48.717741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4cd5e8-a8c9-4b7a-9199-baab23599e0b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:48.720028 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4cd5e8-a8c9-4b7a-9199-baab23599e0b service glance] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:48.731534 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1140/2307] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:49.939619 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f2cbd-74e4-480d-8e57-28aeaad1493e None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:49.940131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f2cbd-74e4-480d-8e57-28aeaad1493e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:49.941303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f2cbd-74e4-480d-8e57-28aeaad1493e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:49.964961 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-107f2cbd-74e4-480d-8e57-28aeaad1493e None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:49.966204 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-107f2cbd-74e4-480d-8e57-28aeaad1493e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:50.030147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1168/2308] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 30 14:58:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:58:50.102416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cd5e8-a8c9-4b7a-9199-baab23599e0b service glance] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:50.104665 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fb4cd5e8-a8c9-4b7a-9199-baab23599e0b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:50.179597 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:50.204704 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cd5e8-a8c9-4b7a-9199-baab23599e0b service glance] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:50.218033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cda71-94b6-42af-905b-b42bdce3a8ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:50.219851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cda71-94b6-42af-905b-b42bdce3a8ef service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:50.221187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cda71-94b6-42af-905b-b42bdce3a8ef service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:50.221761 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cda71-94b6-42af-905b-b42bdce3a8ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:50.236875 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cda71-94b6-42af-905b-b42bdce3a8ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:50.238886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cda71-94b6-42af-905b-b42bdce3a8ef service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:50.248091 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1141/2309] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 14:58:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:58:51.000299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3f1f5-3ec0-40c1-9d6c-c50a79bdfa5d None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:51.000875 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3f1f5-3ec0-40c1-9d6c-c50a79bdfa5d None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:51.000875 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3f1f5-3ec0-40c1-9d6c-c50a79bdfa5d None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:51.002086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1169/2310] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 30 14:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:58:51.047958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c44fe-a3d1-48c2-8dc1-e383b2891f9f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:51.048262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c44fe-a3d1-48c2-8dc1-e383b2891f9f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:51.048505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c44fe-a3d1-48c2-8dc1-e383b2891f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:51.081495 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-d95c44fe-a3d1-48c2-8dc1-e383b2891f9f None None] MFA Rules not processed for user `f4f11655cd51438e9652869f74ffa975`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:51.083144 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d95c44fe-a3d1-48c2-8dc1-e383b2891f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:51.271482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1142/2311] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 30 14:58:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:51.271976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3f1f5-3ec0-40c1-9d6c-c50a79bdfa5d None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:51.271976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-f2d3f1f5-3ec0-40c1-9d6c-c50a79bdfa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:51.271976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3f1f5-3ec0-40c1-9d6c-c50a79bdfa5d None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:51.272243 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5bda6-7e67-4f2c-ab50-dbd2cd15791c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4f11655cd51438e9652869f74ffa975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:51.272243 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5bda6-7e67-4f2c-ab50-dbd2cd15791c service placement] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:51.272243 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5bda6-7e67-4f2c-ab50-dbd2cd15791c service placement] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:51.272243 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5bda6-7e67-4f2c-ab50-dbd2cd15791c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:51.272575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5bda6-7e67-4f2c-ab50-dbd2cd15791c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:51.272575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5bda6-7e67-4f2c-ab50-dbd2cd15791c service placement] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:51.272575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:51.281696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1170/2312] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:52.707271 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c44fe-a3d1-48c2-8dc1-e383b2891f9f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:52.709905 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d95c44fe-a3d1-48c2-8dc1-e383b2891f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:52.812250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c44fe-a3d1-48c2-8dc1-e383b2891f9f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:52.818079 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5bc33-6e62-40fd-8478-a809ab6723b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:52.820766 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5bc33-6e62-40fd-8478-a809ab6723b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:52.820766 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5bc33-6e62-40fd-8478-a809ab6723b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:52.820766 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5bc33-6e62-40fd-8478-a809ab6723b2 admin admin] PATH_INFO: `/v3/users/3fc014703e0b45eb99ed96e9d19a6306` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:52.821068 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5bc33-6e62-40fd-8478-a809ab6723b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fc014703e0b45eb99ed96e9d19a6306)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:52.824208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5bc33-6e62-40fd-8478-a809ab6723b2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:53.065896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-69c5bc33-6e62-40fd-8478-a809ab6723b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fc014703e0b45eb99ed96e9d19a6306'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:53.076447 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-69c5bc33-6e62-40fd-8478-a809ab6723b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fc014703e0b45eb99ed96e9d19a6306'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:53.077561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-69c5bc33-6e62-40fd-8478-a809ab6723b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fc014703e0b45eb99ed96e9d19a6306'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:53.167094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-69c5bc33-6e62-40fd-8478-a809ab6723b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fc014703e0b45eb99ed96e9d19a6306'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:53.190681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:53.194247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-69c5bc33-6e62-40fd-8478-a809ab6723b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fc014703e0b45eb99ed96e9d19a6306'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:53.202759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1143/2313] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:52 2022] DELETE /identity/v3/users/3fc014703e0b45eb99ed96e9d19a6306 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:53.644335 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5bda6-7e67-4f2c-ab50-dbd2cd15791c service placement] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:53.661663 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5bda6-7e67-4f2c-ab50-dbd2cd15791c service placement] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:53.766155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29ff2b-9972-4deb-88db-5a1d4845c15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:53.766855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29ff2b-9972-4deb-88db-5a1d4845c15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:53.767150 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29ff2b-9972-4deb-88db-5a1d4845c15b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:53.767415 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29ff2b-9972-4deb-88db-5a1d4845c15b admin admin] PATH_INFO: `/v3/projects/f60aa51efc8a4c00a10b987fa3465add` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:53.768828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29ff2b-9972-4deb-88db-5a1d4845c15b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f60aa51efc8a4c00a10b987fa3465add)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:53.770746 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29ff2b-9972-4deb-88db-5a1d4845c15b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:53.949134 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ce29ff2b-9972-4deb-88db-5a1d4845c15b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f60aa51efc8a4c00a10b987fa3465add', 'request_id': 'req-ce29ff2b-9972-4deb-88db-5a1d4845c15b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:53.957451 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ce29ff2b-9972-4deb-88db-5a1d4845c15b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f60aa51efc8a4c00a10b987fa3465add', 'request_id': 'req-ce29ff2b-9972-4deb-88db-5a1d4845c15b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:54.034478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ce29ff2b-9972-4deb-88db-5a1d4845c15b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f60aa51efc8a4c00a10b987fa3465add', 'request_id': 'req-ce29ff2b-9972-4deb-88db-5a1d4845c15b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:54.056074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ce29ff2b-9972-4deb-88db-5a1d4845c15b admin admin] The token cache is being invalidate because project f60aa51efc8a4c00a10b987fa3465add was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:54.056275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ce29ff2b-9972-4deb-88db-5a1d4845c15b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:54.058557 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ce29ff2b-9972-4deb-88db-5a1d4845c15b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:54.060473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1171/2314] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:53 2022] DELETE /identity/v3/projects/f60aa51efc8a4c00a10b987fa3465add => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:54.064978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5bc33-6e62-40fd-8478-a809ab6723b2 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:54.067239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-69c5bc33-6e62-40fd-8478-a809ab6723b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:54.089592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5bc33-6e62-40fd-8478-a809ab6723b2 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:54.179655 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84619945-cd0c-4d45-a892-8c4cf3d6c643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:54.181588 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84619945-cd0c-4d45-a892-8c4cf3d6c643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:54.182085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84619945-cd0c-4d45-a892-8c4cf3d6c643 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:54.182354 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84619945-cd0c-4d45-a892-8c4cf3d6c643 admin admin] PATH_INFO: `/v3/users/826ce40c72c646e7bd2ef2fd377b529f` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:54.183745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84619945-cd0c-4d45-a892-8c4cf3d6c643 admin admin] RBAC: Authorizing `identity:delete_user(user_id=826ce40c72c646e7bd2ef2fd377b529f)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:54.185917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84619945-cd0c-4d45-a892-8c4cf3d6c643 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:54.671133 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-84619945-cd0c-4d45-a892-8c4cf3d6c643 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '826ce40c72c646e7bd2ef2fd377b529f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:54.687780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-84619945-cd0c-4d45-a892-8c4cf3d6c643 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '826ce40c72c646e7bd2ef2fd377b529f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:54.688825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-84619945-cd0c-4d45-a892-8c4cf3d6c643 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '826ce40c72c646e7bd2ef2fd377b529f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:54.760277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-84619945-cd0c-4d45-a892-8c4cf3d6c643 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '826ce40c72c646e7bd2ef2fd377b529f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:54.786186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-84619945-cd0c-4d45-a892-8c4cf3d6c643 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '826ce40c72c646e7bd2ef2fd377b529f'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:54.796475 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1144/2315] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:54 2022] DELETE /identity/v3/users/826ce40c72c646e7bd2ef2fd377b529f => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:55.095960 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29ff2b-9972-4deb-88db-5a1d4845c15b admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:55.095960 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ce29ff2b-9972-4deb-88db-5a1d4845c15b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:55.117679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29ff2b-9972-4deb-88db-5a1d4845c15b admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:55.222013 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b74974-8a7d-4654-824b-29bd6a9d065e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:55.222643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b74974-8a7d-4654-824b-29bd6a9d065e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:55.222918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b74974-8a7d-4654-824b-29bd6a9d065e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:55.223283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b74974-8a7d-4654-824b-29bd6a9d065e admin admin] PATH_INFO: `/v3/projects/1748c8178cf84b7b939a6aba4530fa3e` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:55.224931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b74974-8a7d-4654-824b-29bd6a9d065e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1748c8178cf84b7b939a6aba4530fa3e)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:55.226956 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b74974-8a7d-4654-824b-29bd6a9d065e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:55.340107 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:55.356856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-21b74974-8a7d-4654-824b-29bd6a9d065e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1748c8178cf84b7b939a6aba4530fa3e', 'request_id': 'req-21b74974-8a7d-4654-824b-29bd6a9d065e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:55.363072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-21b74974-8a7d-4654-824b-29bd6a9d065e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1748c8178cf84b7b939a6aba4530fa3e', 'request_id': 'req-21b74974-8a7d-4654-824b-29bd6a9d065e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:55.458315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-21b74974-8a7d-4654-824b-29bd6a9d065e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1748c8178cf84b7b939a6aba4530fa3e', 'request_id': 'req-21b74974-8a7d-4654-824b-29bd6a9d065e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:55.478306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-21b74974-8a7d-4654-824b-29bd6a9d065e admin admin] The token cache is being invalidate because project 1748c8178cf84b7b939a6aba4530fa3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:55.478892 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-21b74974-8a7d-4654-824b-29bd6a9d065e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:55.480396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-21b74974-8a7d-4654-824b-29bd6a9d065e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:55.484693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1172/2316] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:55 2022] DELETE /identity/v3/projects/1748c8178cf84b7b939a6aba4530fa3e => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:55.489874 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d009c19-068c-42ec-923a-bae7fb7e8d0d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:55.491476 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d009c19-068c-42ec-923a-bae7fb7e8d0d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:55.491901 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d009c19-068c-42ec-923a-bae7fb7e8d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:55.538107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1d009c19-068c-42ec-923a-bae7fb7e8d0d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:55.540012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1d009c19-068c-42ec-923a-bae7fb7e8d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:55.612675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1145/2317] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:55.620488 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd98c9-b27e-4fd9-a2a1-a2596ae9977d None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:55.620790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd98c9-b27e-4fd9-a2a1-a2596ae9977d None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:55.621074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd98c9-b27e-4fd9-a2a1-a2596ae9977d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:55.663725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-35cd98c9-b27e-4fd9-a2a1-a2596ae9977d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:55.665087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-35cd98c9-b27e-4fd9-a2a1-a2596ae9977d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:55.691703 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1173/2318] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:55.692866 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d009c19-068c-42ec-923a-bae7fb7e8d0d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:55.705568 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d009c19-068c-42ec-923a-bae7fb7e8d0d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:55.711067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a1e6b-7c3d-442d-9377-103358acade5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:55.711907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a1e6b-7c3d-442d-9377-103358acade5 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:55.712288 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a1e6b-7c3d-442d-9377-103358acade5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:55.712639 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a1e6b-7c3d-442d-9377-103358acade5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:55.713384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a1e6b-7c3d-442d-9377-103358acade5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:55.713759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a1e6b-7c3d-442d-9377-103358acade5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:55.715662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a1e6b-7c3d-442d-9377-103358acade5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:55.730008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:55.737625 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1146/2319] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:58:55.739089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd98c9-b27e-4fd9-a2a1-a2596ae9977d None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:55.750330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd98c9-b27e-4fd9-a2a1-a2596ae9977d None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:55.756542 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073056fc-560b-46ee-8bfc-7c7f770e34ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:55.772607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073056fc-560b-46ee-8bfc-7c7f770e34ee admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:55.772607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073056fc-560b-46ee-8bfc-7c7f770e34ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:55.772607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073056fc-560b-46ee-8bfc-7c7f770e34ee admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:55.772607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073056fc-560b-46ee-8bfc-7c7f770e34ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:55.772607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073056fc-560b-46ee-8bfc-7c7f770e34ee admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:55.883822 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a1e6b-7c3d-442d-9377-103358acade5 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:55.885716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1174/2320] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:55 2022] POST /identity/v3/projects => generated 361 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:55.901067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a1e6b-7c3d-442d-9377-103358acade5 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:55.980610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a4f70-9147-4038-aa5b-74be00aad3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:55.981408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a4f70-9147-4038-aa5b-74be00aad3c7 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:55.981710 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a4f70-9147-4038-aa5b-74be00aad3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:55.982845 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a4f70-9147-4038-aa5b-74be00aad3c7 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:55.983318 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575a4f70-9147-4038-aa5b-74be00aad3c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:55.985342 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575a4f70-9147-4038-aa5b-74be00aad3c7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:56.115134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1147/2321] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:55 2022] POST /identity/v3/users => generated 281 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:56.120354 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073056fc-560b-46ee-8bfc-7c7f770e34ee admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:56.134739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073056fc-560b-46ee-8bfc-7c7f770e34ee admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:56.145094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3c543-480a-4ab0-9b77-efcebd3427da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:56.145702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3c543-480a-4ab0-9b77-efcebd3427da admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:56.150620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3c543-480a-4ab0-9b77-efcebd3427da admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:56.150620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3c543-480a-4ab0-9b77-efcebd3427da admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:56.150620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3c543-480a-4ab0-9b77-efcebd3427da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:56.150620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3c543-480a-4ab0-9b77-efcebd3427da admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:56.169236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1175/2322] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:56.170738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a4f70-9147-4038-aa5b-74be00aad3c7 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:56.181200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a4f70-9147-4038-aa5b-74be00aad3c7 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:56.190338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f4ad1-92f4-4683-9ddc-239bbd49e265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:56.190948 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f4ad1-92f4-4683-9ddc-239bbd49e265 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:56.191264 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f4ad1-92f4-4683-9ddc-239bbd49e265 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:56.191539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f4ad1-92f4-4683-9ddc-239bbd49e265 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:56.192202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f4ad1-92f4-4683-9ddc-239bbd49e265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:56.194519 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f4ad1-92f4-4683-9ddc-239bbd49e265 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:56.217335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1148/2323] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:56.219207 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3c543-480a-4ab0-9b77-efcebd3427da admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:56.231942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3c543-480a-4ab0-9b77-efcebd3427da admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:56.245445 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555740b6-2669-4dc7-890b-77966e49f0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:56.246191 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555740b6-2669-4dc7-890b-77966e49f0a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:56.246473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555740b6-2669-4dc7-890b-77966e49f0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:56.246747 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555740b6-2669-4dc7-890b-77966e49f0a6 admin admin] PATH_INFO: `/v3/projects/b7c16dc06e36435e8c32490dfcb7d101/users/9cbe9194583a45e3a136231688b31cf9/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:56.283930 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555740b6-2669-4dc7-890b-77966e49f0a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7c16dc06e36435e8c32490dfcb7d101, user_id=9cbe9194583a45e3a136231688b31cf9, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:56.286623 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555740b6-2669-4dc7-890b-77966e49f0a6 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:56.368751 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1176/2324] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:56 2022] PUT /identity/v3/projects/b7c16dc06e36435e8c32490dfcb7d101/users/9cbe9194583a45e3a136231688b31cf9/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:56.380235 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e0bcb-7294-40e0-bbd6-c5d2728b5a7a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:56.381055 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e0bcb-7294-40e0-bbd6-c5d2728b5a7a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:56.381330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e0bcb-7294-40e0-bbd6-c5d2728b5a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:56.418245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ee0e0bcb-7294-40e0-bbd6-c5d2728b5a7a None None] MFA Rules not processed for user `9cbe9194583a45e3a136231688b31cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:56.421683 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ee0e0bcb-7294-40e0-bbd6-c5d2728b5a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:56.527150 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1149/2325] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:56.562365 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555740b6-2669-4dc7-890b-77966e49f0a6 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:56.564411 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-555740b6-2669-4dc7-890b-77966e49f0a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:56.788163 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555740b6-2669-4dc7-890b-77966e49f0a6 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:56.795648 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21d487-7de4-4f33-8d86-d6aa39f2a952 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:56.840737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21d487-7de4-4f33-8d86-d6aa39f2a952 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:56.840737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21d487-7de4-4f33-8d86-d6aa39f2a952 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:56.840737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21d487-7de4-4f33-8d86-d6aa39f2a952 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:56.840737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21d487-7de4-4f33-8d86-d6aa39f2a952 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:56.840737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21d487-7de4-4f33-8d86-d6aa39f2a952 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:56.853813 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1177/2326] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:56.873643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e0bcb-7294-40e0-bbd6-c5d2728b5a7a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:56.989978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e0bcb-7294-40e0-bbd6-c5d2728b5a7a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:57.000971 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598e6f0-1121-43b4-8a50-9014dce417de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:57.001736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598e6f0-1121-43b4-8a50-9014dce417de admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:57.002073 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598e6f0-1121-43b4-8a50-9014dce417de admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:57.002420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598e6f0-1121-43b4-8a50-9014dce417de admin admin] PATH_INFO: `/v3/users/9cbe9194583a45e3a136231688b31cf9` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:57.004532 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598e6f0-1121-43b4-8a50-9014dce417de admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cbe9194583a45e3a136231688b31cf9)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:57.006900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598e6f0-1121-43b4-8a50-9014dce417de admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:57.023515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:57.328485 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7598e6f0-1121-43b4-8a50-9014dce417de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cbe9194583a45e3a136231688b31cf9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:57.337549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7598e6f0-1121-43b4-8a50-9014dce417de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cbe9194583a45e3a136231688b31cf9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:57.338466 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7598e6f0-1121-43b4-8a50-9014dce417de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cbe9194583a45e3a136231688b31cf9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:57.400874 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7598e6f0-1121-43b4-8a50-9014dce417de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cbe9194583a45e3a136231688b31cf9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:57.414511 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7598e6f0-1121-43b4-8a50-9014dce417de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cbe9194583a45e3a136231688b31cf9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:57.421497 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1150/2327] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:56 2022] DELETE /identity/v3/users/9cbe9194583a45e3a136231688b31cf9 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:57.425947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18957c5-7c3c-4bf3-93c5-7e1a5908ae0b None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:57.426451 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18957c5-7c3c-4bf3-93c5-7e1a5908ae0b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:57.426558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18957c5-7c3c-4bf3-93c5-7e1a5908ae0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:57.451051 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-a18957c5-7c3c-4bf3-93c5-7e1a5908ae0b None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:57.452573 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a18957c5-7c3c-4bf3-93c5-7e1a5908ae0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:57.533146 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:57.546690 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1178/2328] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:57.557015 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598e6f0-1121-43b4-8a50-9014dce417de admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:57.558634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7598e6f0-1121-43b4-8a50-9014dce417de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:57.682529 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598e6f0-1121-43b4-8a50-9014dce417de admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:57.687286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7725d1-685a-4780-92da-02d8a381f0b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:57.687896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7725d1-685a-4780-92da-02d8a381f0b8 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:57.688174 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7725d1-685a-4780-92da-02d8a381f0b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:57.688457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7725d1-685a-4780-92da-02d8a381f0b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:57.702209 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7725d1-685a-4780-92da-02d8a381f0b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:57.703737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7725d1-685a-4780-92da-02d8a381f0b8 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:57.715625 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1151/2329] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:58:58.512680 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18957c5-7c3c-4bf3-93c5-7e1a5908ae0b None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:58.514136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a18957c5-7c3c-4bf3-93c5-7e1a5908ae0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:58.546400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18957c5-7c3c-4bf3-93c5-7e1a5908ae0b None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:58.570650 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b4923-f470-4866-bbad-56bec75a4c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:58.571550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b4923-f470-4866-bbad-56bec75a4c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:58.571891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b4923-f470-4866-bbad-56bec75a4c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:58.572392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b4923-f470-4866-bbad-56bec75a4c73 admin admin] PATH_INFO: `/v3/projects/b7c16dc06e36435e8c32490dfcb7d101` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:58.573710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b4923-f470-4866-bbad-56bec75a4c73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7c16dc06e36435e8c32490dfcb7d101)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:58.575666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b4923-f470-4866-bbad-56bec75a4c73 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:58.708115 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-845b4923-f470-4866-bbad-56bec75a4c73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7c16dc06e36435e8c32490dfcb7d101', 'request_id': 'req-845b4923-f470-4866-bbad-56bec75a4c73'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.720158 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-845b4923-f470-4866-bbad-56bec75a4c73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7c16dc06e36435e8c32490dfcb7d101', 'request_id': 'req-845b4923-f470-4866-bbad-56bec75a4c73'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.789206 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-845b4923-f470-4866-bbad-56bec75a4c73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7c16dc06e36435e8c32490dfcb7d101', 'request_id': 'req-845b4923-f470-4866-bbad-56bec75a4c73'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.811283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-845b4923-f470-4866-bbad-56bec75a4c73 admin admin] The token cache is being invalidate because project b7c16dc06e36435e8c32490dfcb7d101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:58.811283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-845b4923-f470-4866-bbad-56bec75a4c73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.822365 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-845b4923-f470-4866-bbad-56bec75a4c73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:58.831436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1179/2330] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:58 2022] DELETE /identity/v3/projects/b7c16dc06e36435e8c32490dfcb7d101 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:58.832978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b0cbe-0ab2-4292-a19d-012a10bc7dff None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:58.835007 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b0cbe-0ab2-4292-a19d-012a10bc7dff None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:58.835470 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b0cbe-0ab2-4292-a19d-012a10bc7dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:58.863746 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-257b0cbe-0ab2-4292-a19d-012a10bc7dff None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:58.867962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-257b0cbe-0ab2-4292-a19d-012a10bc7dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:58.967198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5faee1-fc29-4e40-b5f1-5eb1fc672f32 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:58.967653 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5faee1-fc29-4e40-b5f1-5eb1fc672f32 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:58.968033 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5faee1-fc29-4e40-b5f1-5eb1fc672f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:58.972949 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1152/2331] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:58.998240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-cd5faee1-fc29-4e40-b5f1-5eb1fc672f32 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:58.999698 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-cd5faee1-fc29-4e40-b5f1-5eb1fc672f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:59.010667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1180/2332] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:59.015147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b0cbe-0ab2-4292-a19d-012a10bc7dff None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:59.029574 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b0cbe-0ab2-4292-a19d-012a10bc7dff None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:59.036995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75620e6a-3bf3-462a-9c85-c9a872b92f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:59.038329 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75620e6a-3bf3-462a-9c85-c9a872b92f81 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.038879 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75620e6a-3bf3-462a-9c85-c9a872b92f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.039224 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75620e6a-3bf3-462a-9c85-c9a872b92f81 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.039864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75620e6a-3bf3-462a-9c85-c9a872b92f81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:58:59.040184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75620e6a-3bf3-462a-9c85-c9a872b92f81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:59.041797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75620e6a-3bf3-462a-9c85-c9a872b92f81 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:59.061716 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1153/2333] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:58:59.062218 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5faee1-fc29-4e40-b5f1-5eb1fc672f32 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:59.090410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5faee1-fc29-4e40-b5f1-5eb1fc672f32 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:59.097776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d43e5-0402-4b4b-9329-abe0916a29aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:59.098642 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d43e5-0402-4b4b-9329-abe0916a29aa admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.100732 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d43e5-0402-4b4b-9329-abe0916a29aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.101087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d43e5-0402-4b4b-9329-abe0916a29aa admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.101885 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d43e5-0402-4b4b-9329-abe0916a29aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:59.104363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d43e5-0402-4b4b-9329-abe0916a29aa admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:59.223487 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1181/2334] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:59 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:58:59.229547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75620e6a-3bf3-462a-9c85-c9a872b92f81 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:59.251585 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75620e6a-3bf3-462a-9c85-c9a872b92f81 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:59.282019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:59.356882 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abbb7e-a130-44d4-a706-98f81662497f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:59.357736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abbb7e-a130-44d4-a706-98f81662497f admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.358539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abbb7e-a130-44d4-a706-98f81662497f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.359005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abbb7e-a130-44d4-a706-98f81662497f admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.360168 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5abbb7e-a130-44d4-a706-98f81662497f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:59.362307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5abbb7e-a130-44d4-a706-98f81662497f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:59.476691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1154/2335] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:58:59 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:58:59.481243 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d43e5-0402-4b4b-9329-abe0916a29aa admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:59.501709 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d43e5-0402-4b4b-9329-abe0916a29aa admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:59.509066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8596c43-8ad1-4a4c-8af0-43bfeae30ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:59.512403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8596c43-8ad1-4a4c-8af0-43bfeae30ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.512900 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8596c43-8ad1-4a4c-8af0-43bfeae30ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.514125 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8596c43-8ad1-4a4c-8af0-43bfeae30ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.514961 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8596c43-8ad1-4a4c-8af0-43bfeae30ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:59.516837 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8596c43-8ad1-4a4c-8af0-43bfeae30ea4 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:59.536398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abbb7e-a130-44d4-a706-98f81662497f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:59.546184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1182/2336] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:59.551233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abbb7e-a130-44d4-a706-98f81662497f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:59.564734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9f50d-de3d-4320-a889-ac5f763d2d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:59.565362 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9f50d-de3d-4320-a889-ac5f763d2d4c admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.565640 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9f50d-de3d-4320-a889-ac5f763d2d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.565912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9f50d-de3d-4320-a889-ac5f763d2d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.566565 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9f50d-de3d-4320-a889-ac5f763d2d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:59.569561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9f50d-de3d-4320-a889-ac5f763d2d4c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:59.590823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8596c43-8ad1-4a4c-8af0-43bfeae30ea4 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:59.591516 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1155/2337] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:58:59.602294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8596c43-8ad1-4a4c-8af0-43bfeae30ea4 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:59.608201 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db18658-d347-4f30-ba2a-70cf72e128cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:59.609013 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db18658-d347-4f30-ba2a-70cf72e128cc admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.609309 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db18658-d347-4f30-ba2a-70cf72e128cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.609902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db18658-d347-4f30-ba2a-70cf72e128cc admin admin] PATH_INFO: `/v3/projects/9df86f8788394b47802b8667cb725d47/users/62bc662b15154721bc741c9d8f048d44/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.629159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db18658-d347-4f30-ba2a-70cf72e128cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df86f8788394b47802b8667cb725d47, user_id=62bc662b15154721bc741c9d8f048d44, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:59.633216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db18658-d347-4f30-ba2a-70cf72e128cc admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:59.747571 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1183/2338] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:58:59 2022] PUT /identity/v3/projects/9df86f8788394b47802b8667cb725d47/users/62bc662b15154721bc741c9d8f048d44/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:58:59.758121 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e29bb0-c82b-4527-9fa5-b89553590854 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.758448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e29bb0-c82b-4527-9fa5-b89553590854 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.758677 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e29bb0-c82b-4527-9fa5-b89553590854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.794389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-99e29bb0-c82b-4527-9fa5-b89553590854 None None] MFA Rules not processed for user `62bc662b15154721bc741c9d8f048d44`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:59.796727 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-99e29bb0-c82b-4527-9fa5-b89553590854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:59.891921 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1156/2339] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:58:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:59.896468 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db18658-d347-4f30-ba2a-70cf72e128cc admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:59.984283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db18658-d347-4f30-ba2a-70cf72e128cc admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:59.989476 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2e614-6291-40dd-aac1-b65d404b74e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:59.995011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2e614-6291-40dd-aac1-b65d404b74e3 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:59.995011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2e614-6291-40dd-aac1-b65d404b74e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:59.995011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2e614-6291-40dd-aac1-b65d404b74e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:59.995011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2e614-6291-40dd-aac1-b65d404b74e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:59.995011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2e614-6291-40dd-aac1-b65d404b74e3 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:00.134520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1184/2340] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:58:59 2022] POST /identity/v3/projects => generated 385 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:00.138357 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e29bb0-c82b-4527-9fa5-b89553590854 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:00.150417 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e29bb0-c82b-4527-9fa5-b89553590854 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:00.220482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b758c0-d748-4797-b0e9-124ac73d268c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:00.221011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b758c0-d748-4797-b0e9-124ac73d268c admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:00.221434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b758c0-d748-4797-b0e9-124ac73d268c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:00.221710 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b758c0-d748-4797-b0e9-124ac73d268c admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:00.222356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b758c0-d748-4797-b0e9-124ac73d268c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:00.225199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b758c0-d748-4797-b0e9-124ac73d268c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:00.349529 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1157/2341] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:59:00 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:00.355379 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2e614-6291-40dd-aac1-b65d404b74e3 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:00.363119 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:00.368617 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2e614-6291-40dd-aac1-b65d404b74e3 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:00.373719 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875ee49-5f23-431a-ab67-edb4ff8a0a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:00.374360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875ee49-5f23-431a-ab67-edb4ff8a0a62 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:00.374640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875ee49-5f23-431a-ab67-edb4ff8a0a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:00.374911 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875ee49-5f23-431a-ab67-edb4ff8a0a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:00.375622 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875ee49-5f23-431a-ab67-edb4ff8a0a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:00.379652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875ee49-5f23-431a-ab67-edb4ff8a0a62 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:00.393934 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1185/2342] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:00.398599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b758c0-d748-4797-b0e9-124ac73d268c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:00.425801 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b758c0-d748-4797-b0e9-124ac73d268c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:00.430220 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191438d-94e1-4415-886a-9b7a713f02ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:00.430828 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191438d-94e1-4415-886a-9b7a713f02ed admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:00.431125 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191438d-94e1-4415-886a-9b7a713f02ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:00.431422 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191438d-94e1-4415-886a-9b7a713f02ed admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:00.432332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191438d-94e1-4415-886a-9b7a713f02ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:00.434122 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191438d-94e1-4415-886a-9b7a713f02ed admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:00.449064 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1158/2343] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:00.452279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875ee49-5f23-431a-ab67-edb4ff8a0a62 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:00.464149 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875ee49-5f23-431a-ab67-edb4ff8a0a62 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:00.476439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6c5fb-9eb6-4f3c-afda-13ed4856aa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:00.477184 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6c5fb-9eb6-4f3c-afda-13ed4856aa99 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:00.477527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6c5fb-9eb6-4f3c-afda-13ed4856aa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:00.477873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6c5fb-9eb6-4f3c-afda-13ed4856aa99 admin admin] PATH_INFO: `/v3/projects/bc1e57cca8e74e599912a56740dec03d/users/6737a6a8c1d447c7aa6c0daf65093d4b/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:00.501049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6c5fb-9eb6-4f3c-afda-13ed4856aa99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc1e57cca8e74e599912a56740dec03d, user_id=6737a6a8c1d447c7aa6c0daf65093d4b, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:00.503884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6c5fb-9eb6-4f3c-afda-13ed4856aa99 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:00.600067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1186/2344] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:00 2022] PUT /identity/v3/projects/bc1e57cca8e74e599912a56740dec03d/users/6737a6a8c1d447c7aa6c0daf65093d4b/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:00.684531 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fa705-484d-465a-9e60-c1b10208d203 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:00.684810 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fa705-484d-465a-9e60-c1b10208d203 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:00.685087 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fa705-484d-465a-9e60-c1b10208d203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:00.694975 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:00.745045 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e84fa705-484d-465a-9e60-c1b10208d203 None None] MFA Rules not processed for user `6737a6a8c1d447c7aa6c0daf65093d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:00.746700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e84fa705-484d-465a-9e60-c1b10208d203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:00.879969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1159/2345] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:59:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:59:00.885992 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6c5fb-9eb6-4f3c-afda-13ed4856aa99 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:00.994113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6c5fb-9eb6-4f3c-afda-13ed4856aa99 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:01.000902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb566b2-c651-4f03-9ce7-29e3e4003b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:01.002291 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb566b2-c651-4f03-9ce7-29e3e4003b1c admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:01.002637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb566b2-c651-4f03-9ce7-29e3e4003b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:01.003192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb566b2-c651-4f03-9ce7-29e3e4003b1c admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:01.004216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb566b2-c651-4f03-9ce7-29e3e4003b1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:01.006252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb566b2-c651-4f03-9ce7-29e3e4003b1c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:01.105651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1187/2346] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:59:00 2022] POST /identity/v3/projects => generated 385 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:01.111325 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84fa705-484d-465a-9e60-c1b10208d203 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:01.125540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84fa705-484d-465a-9e60-c1b10208d203 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:01.242312 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b3e0c-28aa-45d3-bc22-b048a41ca06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:01.242312 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b3e0c-28aa-45d3-bc22-b048a41ca06f admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:01.242312 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b3e0c-28aa-45d3-bc22-b048a41ca06f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:01.242312 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b3e0c-28aa-45d3-bc22-b048a41ca06f admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:01.242710 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b3e0c-28aa-45d3-bc22-b048a41ca06f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:01.242710 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b3e0c-28aa-45d3-bc22-b048a41ca06f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:01.911340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1160/2347] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:59:01 2022] POST /identity/v3/users => generated 299 bytes in 800 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:59:01.914987 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb566b2-c651-4f03-9ce7-29e3e4003b1c admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:01.925739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb566b2-c651-4f03-9ce7-29e3e4003b1c admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:01.930927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d2e02-2f94-4940-abb8-15397df5b761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:01.931559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d2e02-2f94-4940-abb8-15397df5b761 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:01.931836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d2e02-2f94-4940-abb8-15397df5b761 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:01.932105 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d2e02-2f94-4940-abb8-15397df5b761 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:01.932745 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d2e02-2f94-4940-abb8-15397df5b761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:01.934333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d2e02-2f94-4940-abb8-15397df5b761 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:01.949556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1188/2348] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:01.951781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b3e0c-28aa-45d3-bc22-b048a41ca06f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:01.966588 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b3e0c-28aa-45d3-bc22-b048a41ca06f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:01.972262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3d312-b12b-4648-a139-9c1bbe4ddb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:01.974050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3d312-b12b-4648-a139-9c1bbe4ddb36 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:01.974050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3d312-b12b-4648-a139-9c1bbe4ddb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:01.974050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3d312-b12b-4648-a139-9c1bbe4ddb36 admin admin] PATH_INFO: `/v3/projects/a6d8889720db49efacc15b4a12f8bd25/users/ef896d5dfb5d45e8b652b173e39764a1/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:01.992332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3d312-b12b-4648-a139-9c1bbe4ddb36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6d8889720db49efacc15b4a12f8bd25, user_id=ef896d5dfb5d45e8b652b173e39764a1, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:01.995594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3d312-b12b-4648-a139-9c1bbe4ddb36 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:02.109397 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1161/2349] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:01 2022] PUT /identity/v3/projects/a6d8889720db49efacc15b4a12f8bd25/users/ef896d5dfb5d45e8b652b173e39764a1/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:02.113690 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d2e02-2f94-4940-abb8-15397df5b761 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:02.129014 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d2e02-2f94-4940-abb8-15397df5b761 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:02.218938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afd36a-5222-4e56-80fb-449b700380e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:02.219696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afd36a-5222-4e56-80fb-449b700380e4 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:02.220167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afd36a-5222-4e56-80fb-449b700380e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:02.220646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afd36a-5222-4e56-80fb-449b700380e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:02.221426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afd36a-5222-4e56-80fb-449b700380e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:02.223678 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afd36a-5222-4e56-80fb-449b700380e4 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:02.244993 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1189/2350] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:02.246800 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3d312-b12b-4648-a139-9c1bbe4ddb36 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:02.268480 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3d312-b12b-4648-a139-9c1bbe4ddb36 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:02.282087 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253738c-94d0-4d33-a3ce-cca3962c4ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:02.283321 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253738c-94d0-4d33-a3ce-cca3962c4ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:02.289449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253738c-94d0-4d33-a3ce-cca3962c4ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:02.289449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253738c-94d0-4d33-a3ce-cca3962c4ffe admin admin] PATH_INFO: `/v3/projects/a6d8889720db49efacc15b4a12f8bd25/users/ef896d5dfb5d45e8b652b173e39764a1/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:02.289449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253738c-94d0-4d33-a3ce-cca3962c4ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6d8889720db49efacc15b4a12f8bd25, user_id=ef896d5dfb5d45e8b652b173e39764a1, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:02.291420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253738c-94d0-4d33-a3ce-cca3962c4ffe admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:02.304137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1162/2351] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:02 2022] PUT /identity/v3/projects/a6d8889720db49efacc15b4a12f8bd25/users/ef896d5dfb5d45e8b652b173e39764a1/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:02.315992 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75b170-cdaa-4141-9394-9c10eeac5783 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:02.316549 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75b170-cdaa-4141-9394-9c10eeac5783 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:02.316826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75b170-cdaa-4141-9394-9c10eeac5783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:02.355782 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-3e75b170-cdaa-4141-9394-9c10eeac5783 None None] MFA Rules not processed for user `ef896d5dfb5d45e8b652b173e39764a1`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:02.357093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-3e75b170-cdaa-4141-9394-9c10eeac5783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:02.391043 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:02.475236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1190/2352] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:59:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:02.486354 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253738c-94d0-4d33-a3ce-cca3962c4ffe admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:02.488128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4253738c-94d0-4d33-a3ce-cca3962c4ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:02.575555 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:02.581343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253738c-94d0-4d33-a3ce-cca3962c4ffe admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:02.585439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d4b9f-54ef-4b7b-b428-6d21908ecfb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:02.587760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d4b9f-54ef-4b7b-b428-6d21908ecfb5 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:02.587760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d4b9f-54ef-4b7b-b428-6d21908ecfb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:02.587760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d4b9f-54ef-4b7b-b428-6d21908ecfb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:02.650194 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d4b9f-54ef-4b7b-b428-6d21908ecfb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:02.651990 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d4b9f-54ef-4b7b-b428-6d21908ecfb5 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:02.662572 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1163/2353] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:04.819645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75b170-cdaa-4141-9394-9c10eeac5783 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:04.835453 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75b170-cdaa-4141-9394-9c10eeac5783 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:04.844093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449e218-cdb5-438b-8b10-2e6c7b8289b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:04.844866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449e218-cdb5-438b-8b10-2e6c7b8289b2 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:04.845169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449e218-cdb5-438b-8b10-2e6c7b8289b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:04.845443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449e218-cdb5-438b-8b10-2e6c7b8289b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:04.975696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449e218-cdb5-438b-8b10-2e6c7b8289b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:04.977213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449e218-cdb5-438b-8b10-2e6c7b8289b2 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:05.008740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1191/2354] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:06.541610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d4b9f-54ef-4b7b-b428-6d21908ecfb5 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:06.550153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d4b9f-54ef-4b7b-b428-6d21908ecfb5 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:06.554279 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f81439-16b0-4a3d-85da-7ea69b273e89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:06.554930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f81439-16b0-4a3d-85da-7ea69b273e89 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:06.555255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f81439-16b0-4a3d-85da-7ea69b273e89 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:06.555544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f81439-16b0-4a3d-85da-7ea69b273e89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:06.564626 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f81439-16b0-4a3d-85da-7ea69b273e89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:06.566058 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f81439-16b0-4a3d-85da-7ea69b273e89 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:06.576223 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1164/2355] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:10.379272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b61b5d-f64c-419d-9fe1-01e31b8aa5b6 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:10.379770 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b61b5d-f64c-419d-9fe1-01e31b8aa5b6 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:10.379959 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b61b5d-f64c-419d-9fe1-01e31b8aa5b6 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:10.381320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1192/2356] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 30 14:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:59:10.387128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac099f78-de47-47ef-b974-a3696226d079 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:10.387579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac099f78-de47-47ef-b974-a3696226d079 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:10.387976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac099f78-de47-47ef-b974-a3696226d079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:10.407864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ac099f78-de47-47ef-b974-a3696226d079 None None] MFA Rules not processed for user `5b6e4c5bf2a54b28a55e5ae74b90ab4f`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:10.409189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ac099f78-de47-47ef-b974-a3696226d079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:10.486352 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1165/2357] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 30 14:59:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:10.496276 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b61b5d-f64c-419d-9fe1-01e31b8aa5b6 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:10.498037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-42b61b5d-f64c-419d-9fe1-01e31b8aa5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:10.557942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b61b5d-f64c-419d-9fe1-01e31b8aa5b6 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:10.561424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391041c-53b9-4b47-932f-c087f7a70a6a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfda7307e703427faaf599e32b91bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:10.562011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391041c-53b9-4b47-932f-c087f7a70a6a service glance] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:10.562282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391041c-53b9-4b47-932f-c087f7a70a6a service glance] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:10.562549 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391041c-53b9-4b47-932f-c087f7a70a6a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:10.573153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391041c-53b9-4b47-932f-c087f7a70a6a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:10.574648 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391041c-53b9-4b47-932f-c087f7a70a6a service glance] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:10.585656 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1193/2358] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:10.818919 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3530a0-69e6-44a8-af7e-982b194452c0 None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:10.819141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3530a0-69e6-44a8-af7e-982b194452c0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:10.819410 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3530a0-69e6-44a8-af7e-982b194452c0 None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:10.820810 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1166/2359] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 30 14:59:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:59:10.825517 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f1abb-47a3-4cbc-979c-74005c69aaed None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:10.825796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f1abb-47a3-4cbc-979c-74005c69aaed None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:10.826067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f1abb-47a3-4cbc-979c-74005c69aaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:10.845360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e33f1abb-47a3-4cbc-979c-74005c69aaed None None] MFA Rules not processed for user `7bff0664e0b44b7c85344a9f3e88f126`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:10.846573 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e33f1abb-47a3-4cbc-979c-74005c69aaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:10.897002 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1194/2360] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 30 14:59:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:10.908754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3530a0-69e6-44a8-af7e-982b194452c0 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:10.910404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bf3530a0-69e6-44a8-af7e-982b194452c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:10.923254 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3530a0-69e6-44a8-af7e-982b194452c0 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:10.927218 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc543483-ece2-419a-b6b7-58031564207d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:10.927796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc543483-ece2-419a-b6b7-58031564207d service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:10.928067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc543483-ece2-419a-b6b7-58031564207d service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:10.928332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc543483-ece2-419a-b6b7-58031564207d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:10.935039 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc543483-ece2-419a-b6b7-58031564207d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:10.936500 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc543483-ece2-419a-b6b7-58031564207d service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:10.944637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1167/2361] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:11.030393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4edaa-93b7-41c6-ade2-620a0d57aa95 None None] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:11.030680 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4edaa-93b7-41c6-ade2-620a0d57aa95 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:11.030956 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4edaa-93b7-41c6-ade2-620a0d57aa95 None None] PATH_INFO: `/` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:11.032319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1195/2362] 10.222.0.27 () {54 vars in 921 bytes} [Tue Aug 30 14:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:59:11.038070 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26186e70-0fef-4824-a633-49d1abf0864d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:11.038349 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26186e70-0fef-4824-a633-49d1abf0864d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:11.038616 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26186e70-0fef-4824-a633-49d1abf0864d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:11.058299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-26186e70-0fef-4824-a633-49d1abf0864d None None] MFA Rules not processed for user `5b6e4c5bf2a54b28a55e5ae74b90ab4f`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:11.059503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-26186e70-0fef-4824-a633-49d1abf0864d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:11.069707 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1168/2363] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Aug 30 14:59:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:59:11.079830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4edaa-93b7-41c6-ade2-620a0d57aa95 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:11.081526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8ba4edaa-93b7-41c6-ade2-620a0d57aa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:11.094566 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4edaa-93b7-41c6-ade2-620a0d57aa95 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:11.098147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdd1fd-19e0-4b5f-ada0-50b8dd35e2de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:11.099321 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdd1fd-19e0-4b5f-ada0-50b8dd35e2de service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:11.099602 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdd1fd-19e0-4b5f-ada0-50b8dd35e2de service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:11.099871 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdd1fd-19e0-4b5f-ada0-50b8dd35e2de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:11.110075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bdd1fd-19e0-4b5f-ada0-50b8dd35e2de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:11.111538 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bdd1fd-19e0-4b5f-ada0-50b8dd35e2de service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:11.117828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:11.123024 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1196/2364] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:15.505733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26186e70-0fef-4824-a633-49d1abf0864d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:15.567466 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26186e70-0fef-4824-a633-49d1abf0864d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:15.571985 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658d440-0853-4830-8ff6-2067d5476e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:15.572606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658d440-0853-4830-8ff6-2067d5476e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:15.572885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658d440-0853-4830-8ff6-2067d5476e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:15.573155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658d440-0853-4830-8ff6-2067d5476e37 admin admin] PATH_INFO: `/v3/users/62bc662b15154721bc741c9d8f048d44` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:15.574266 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658d440-0853-4830-8ff6-2067d5476e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62bc662b15154721bc741c9d8f048d44)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:15.576284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658d440-0853-4830-8ff6-2067d5476e37 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:15.577749 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:15.728039 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3658d440-0853-4830-8ff6-2067d5476e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62bc662b15154721bc741c9d8f048d44'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:15.734944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3658d440-0853-4830-8ff6-2067d5476e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62bc662b15154721bc741c9d8f048d44'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:15.736242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3658d440-0853-4830-8ff6-2067d5476e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62bc662b15154721bc741c9d8f048d44'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:15.782077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3658d440-0853-4830-8ff6-2067d5476e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62bc662b15154721bc741c9d8f048d44'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:15.794885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-3658d440-0853-4830-8ff6-2067d5476e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62bc662b15154721bc741c9d8f048d44'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:15.801393 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1169/2365] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:59:15 2022] DELETE /identity/v3/users/62bc662b15154721bc741c9d8f048d44 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:15.805639 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb817a65-9e01-46e6-a2f3-2c8de7e3b1f0 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:15.806078 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb817a65-9e01-46e6-a2f3-2c8de7e3b1f0 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:15.806190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb817a65-9e01-46e6-a2f3-2c8de7e3b1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:15.825268 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-fb817a65-9e01-46e6-a2f3-2c8de7e3b1f0 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:15.826476 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-fb817a65-9e01-46e6-a2f3-2c8de7e3b1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:15.892074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1197/2366] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:15.901930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658d440-0853-4830-8ff6-2067d5476e37 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:15.903352 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3658d440-0853-4830-8ff6-2067d5476e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:15.963087 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658d440-0853-4830-8ff6-2067d5476e37 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:15.967017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c3e9b-401b-4ae3-8008-c40c585e2716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:15.967670 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c3e9b-401b-4ae3-8008-c40c585e2716 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:15.968018 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c3e9b-401b-4ae3-8008-c40c585e2716 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:15.968303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c3e9b-401b-4ae3-8008-c40c585e2716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:15.979572 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c3e9b-401b-4ae3-8008-c40c585e2716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:15.981158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c3e9b-401b-4ae3-8008-c40c585e2716 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:15.993324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1170/2367] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:16.459942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb817a65-9e01-46e6-a2f3-2c8de7e3b1f0 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:16.464168 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-fb817a65-9e01-46e6-a2f3-2c8de7e3b1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:16.493866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb817a65-9e01-46e6-a2f3-2c8de7e3b1f0 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:16.515211 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc078f4-b3ee-4485-97f7-6007bbef70f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:16.516040 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc078f4-b3ee-4485-97f7-6007bbef70f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:16.516419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc078f4-b3ee-4485-97f7-6007bbef70f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:16.516782 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc078f4-b3ee-4485-97f7-6007bbef70f8 admin admin] PATH_INFO: `/v3/projects/9df86f8788394b47802b8667cb725d47` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:16.518127 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc078f4-b3ee-4485-97f7-6007bbef70f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df86f8788394b47802b8667cb725d47)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:16.520173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc078f4-b3ee-4485-97f7-6007bbef70f8 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:16.639306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-bfc078f4-b3ee-4485-97f7-6007bbef70f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df86f8788394b47802b8667cb725d47', 'request_id': 'req-bfc078f4-b3ee-4485-97f7-6007bbef70f8'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:16.644666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-bfc078f4-b3ee-4485-97f7-6007bbef70f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df86f8788394b47802b8667cb725d47', 'request_id': 'req-bfc078f4-b3ee-4485-97f7-6007bbef70f8'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:16.720018 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-bfc078f4-b3ee-4485-97f7-6007bbef70f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df86f8788394b47802b8667cb725d47', 'request_id': 'req-bfc078f4-b3ee-4485-97f7-6007bbef70f8'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:16.739713 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-bfc078f4-b3ee-4485-97f7-6007bbef70f8 admin admin] The token cache is being invalidate because project 9df86f8788394b47802b8667cb725d47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:59:16.740065 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-bfc078f4-b3ee-4485-97f7-6007bbef70f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:16.740941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-bfc078f4-b3ee-4485-97f7-6007bbef70f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:16.742888 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1198/2368] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:59:16 2022] DELETE /identity/v3/projects/9df86f8788394b47802b8667cb725d47 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:16.746548 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c3e9b-401b-4ae3-8008-c40c585e2716 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:16.756523 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c3e9b-401b-4ae3-8008-c40c585e2716 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:16.816873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8f54d-c354-459a-8a76-9bfbd4dd9582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:16.818545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8f54d-c354-459a-8a76-9bfbd4dd9582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:16.818545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8f54d-c354-459a-8a76-9bfbd4dd9582 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:16.818545 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8f54d-c354-459a-8a76-9bfbd4dd9582 admin admin] PATH_INFO: `/v3/users/6737a6a8c1d447c7aa6c0daf65093d4b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:16.819704 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8f54d-c354-459a-8a76-9bfbd4dd9582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6737a6a8c1d447c7aa6c0daf65093d4b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:16.821700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8f54d-c354-459a-8a76-9bfbd4dd9582 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:16.847965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:17.036686 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b2c8f54d-c354-459a-8a76-9bfbd4dd9582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6737a6a8c1d447c7aa6c0daf65093d4b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.045446 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b2c8f54d-c354-459a-8a76-9bfbd4dd9582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6737a6a8c1d447c7aa6c0daf65093d4b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.046137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b2c8f54d-c354-459a-8a76-9bfbd4dd9582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6737a6a8c1d447c7aa6c0daf65093d4b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.101887 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b2c8f54d-c354-459a-8a76-9bfbd4dd9582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6737a6a8c1d447c7aa6c0daf65093d4b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.116435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b2c8f54d-c354-459a-8a76-9bfbd4dd9582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6737a6a8c1d447c7aa6c0daf65093d4b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.122748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1171/2369] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:59:16 2022] DELETE /identity/v3/users/6737a6a8c1d447c7aa6c0daf65093d4b => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:17.549436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc078f4-b3ee-4485-97f7-6007bbef70f8 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:17.550852 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-bfc078f4-b3ee-4485-97f7-6007bbef70f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:17.572731 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc078f4-b3ee-4485-97f7-6007bbef70f8 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:17.636674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238d6cf-7257-4eeb-9bf7-681739a25b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:17.637416 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238d6cf-7257-4eeb-9bf7-681739a25b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:17.637775 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238d6cf-7257-4eeb-9bf7-681739a25b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:17.638044 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238d6cf-7257-4eeb-9bf7-681739a25b7b admin admin] PATH_INFO: `/v3/projects/bc1e57cca8e74e599912a56740dec03d` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:17.639198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238d6cf-7257-4eeb-9bf7-681739a25b7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc1e57cca8e74e599912a56740dec03d)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:17.641102 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238d6cf-7257-4eeb-9bf7-681739a25b7b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:17.726427 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:17.743155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-3238d6cf-7257-4eeb-9bf7-681739a25b7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc1e57cca8e74e599912a56740dec03d', 'request_id': 'req-3238d6cf-7257-4eeb-9bf7-681739a25b7b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.748569 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-3238d6cf-7257-4eeb-9bf7-681739a25b7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc1e57cca8e74e599912a56740dec03d', 'request_id': 'req-3238d6cf-7257-4eeb-9bf7-681739a25b7b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.867736 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-3238d6cf-7257-4eeb-9bf7-681739a25b7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc1e57cca8e74e599912a56740dec03d', 'request_id': 'req-3238d6cf-7257-4eeb-9bf7-681739a25b7b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.879164 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-3238d6cf-7257-4eeb-9bf7-681739a25b7b admin admin] The token cache is being invalidate because project bc1e57cca8e74e599912a56740dec03d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:59:17.879455 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-3238d6cf-7257-4eeb-9bf7-681739a25b7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.880083 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-3238d6cf-7257-4eeb-9bf7-681739a25b7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:17.881547 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1199/2370] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:59:17 2022] DELETE /identity/v3/projects/bc1e57cca8e74e599912a56740dec03d => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:59:17.885539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8f54d-c354-459a-8a76-9bfbd4dd9582 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:17.886949 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b2c8f54d-c354-459a-8a76-9bfbd4dd9582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:17.899600 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8f54d-c354-459a-8a76-9bfbd4dd9582 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:17.958683 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09efd9-313f-4d64-8a1f-eda43e4c545c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:17.959344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09efd9-313f-4d64-8a1f-eda43e4c545c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:17.959626 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09efd9-313f-4d64-8a1f-eda43e4c545c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:17.959898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09efd9-313f-4d64-8a1f-eda43e4c545c admin admin] PATH_INFO: `/v3/users/ef896d5dfb5d45e8b652b173e39764a1` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:17.961017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09efd9-313f-4d64-8a1f-eda43e4c545c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef896d5dfb5d45e8b652b173e39764a1)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:17.962944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09efd9-313f-4d64-8a1f-eda43e4c545c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:18.138448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cb09efd9-313f-4d64-8a1f-eda43e4c545c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef896d5dfb5d45e8b652b173e39764a1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:18.145979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cb09efd9-313f-4d64-8a1f-eda43e4c545c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef896d5dfb5d45e8b652b173e39764a1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:18.146657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cb09efd9-313f-4d64-8a1f-eda43e4c545c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef896d5dfb5d45e8b652b173e39764a1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:18.238913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cb09efd9-313f-4d64-8a1f-eda43e4c545c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef896d5dfb5d45e8b652b173e39764a1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:18.252085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cb09efd9-313f-4d64-8a1f-eda43e4c545c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef896d5dfb5d45e8b652b173e39764a1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:18.258070 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1172/2371] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:59:17 2022] DELETE /identity/v3/users/ef896d5dfb5d45e8b652b173e39764a1 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:18.648265 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238d6cf-7257-4eeb-9bf7-681739a25b7b admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:18.650952 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-3238d6cf-7257-4eeb-9bf7-681739a25b7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:18.668074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238d6cf-7257-4eeb-9bf7-681739a25b7b admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:18.728177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe789de-6ead-4b44-8df1-b448b23887fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:18.728798 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe789de-6ead-4b44-8df1-b448b23887fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:18.729077 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe789de-6ead-4b44-8df1-b448b23887fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:18.729351 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe789de-6ead-4b44-8df1-b448b23887fd admin admin] PATH_INFO: `/v3/projects/a6d8889720db49efacc15b4a12f8bd25` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:18.730492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe789de-6ead-4b44-8df1-b448b23887fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6d8889720db49efacc15b4a12f8bd25)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:18.732443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe789de-6ead-4b44-8df1-b448b23887fd admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:18.867112 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9fe789de-6ead-4b44-8df1-b448b23887fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6d8889720db49efacc15b4a12f8bd25', 'request_id': 'req-9fe789de-6ead-4b44-8df1-b448b23887fd'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:18.872594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9fe789de-6ead-4b44-8df1-b448b23887fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6d8889720db49efacc15b4a12f8bd25', 'request_id': 'req-9fe789de-6ead-4b44-8df1-b448b23887fd'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:18.929940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9fe789de-6ead-4b44-8df1-b448b23887fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6d8889720db49efacc15b4a12f8bd25', 'request_id': 'req-9fe789de-6ead-4b44-8df1-b448b23887fd'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:18.941071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9fe789de-6ead-4b44-8df1-b448b23887fd admin admin] The token cache is being invalidate because project a6d8889720db49efacc15b4a12f8bd25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:59:18.949884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9fe789de-6ead-4b44-8df1-b448b23887fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:18.949884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9fe789de-6ead-4b44-8df1-b448b23887fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:18.949884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1200/2372] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:59:18 2022] DELETE /identity/v3/projects/a6d8889720db49efacc15b4a12f8bd25 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:20.064511 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65945b-1f65-4c4b-b674-ecc4a28cab64 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:20.064982 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65945b-1f65-4c4b-b674-ecc4a28cab64 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:20.065070 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65945b-1f65-4c4b-b674-ecc4a28cab64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:20.085928 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ef65945b-1f65-4c4b-b674-ecc4a28cab64 None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:20.087201 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ef65945b-1f65-4c4b-b674-ecc4a28cab64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:20.100961 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:20.143807 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1173/2373] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 30 14:59:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:20.155760 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe789de-6ead-4b44-8df1-b448b23887fd admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:20.157149 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9fe789de-6ead-4b44-8df1-b448b23887fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:20.204881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:20.228719 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe789de-6ead-4b44-8df1-b448b23887fd admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:20.233926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd72b5-3072-4cac-b089-955668e657fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:20.234545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd72b5-3072-4cac-b089-955668e657fa service swift] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:20.234824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd72b5-3072-4cac-b089-955668e657fa service swift] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:20.235130 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd72b5-3072-4cac-b089-955668e657fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:20.244602 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd72b5-3072-4cac-b089-955668e657fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:20.246270 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd72b5-3072-4cac-b089-955668e657fa service swift] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:20.255072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1201/2374] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 14:59:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:59:28.409228 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65945b-1f65-4c4b-b674-ecc4a28cab64 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:28.410856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ef65945b-1f65-4c4b-b674-ecc4a28cab64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:28.486345 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65945b-1f65-4c4b-b674-ecc4a28cab64 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:28.491047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c3089-3637-4ad1-bbc7-59b74b3ea01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:28.491695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c3089-3637-4ad1-bbc7-59b74b3ea01e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:28.491969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c3089-3637-4ad1-bbc7-59b74b3ea01e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:28.492237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c3089-3637-4ad1-bbc7-59b74b3ea01e admin admin] PATH_INFO: `/v3/users/81b90c27ef274539895af6a0918cc0dd` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:28.493364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c3089-3637-4ad1-bbc7-59b74b3ea01e admin admin] RBAC: Authorizing `identity:delete_user(user_id=81b90c27ef274539895af6a0918cc0dd)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:28.495313 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c3089-3637-4ad1-bbc7-59b74b3ea01e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:28.697660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-098c3089-3637-4ad1-bbc7-59b74b3ea01e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81b90c27ef274539895af6a0918cc0dd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:28.705183 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-098c3089-3637-4ad1-bbc7-59b74b3ea01e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81b90c27ef274539895af6a0918cc0dd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:28.705980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-098c3089-3637-4ad1-bbc7-59b74b3ea01e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81b90c27ef274539895af6a0918cc0dd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:28.792849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-098c3089-3637-4ad1-bbc7-59b74b3ea01e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81b90c27ef274539895af6a0918cc0dd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:28.806408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-098c3089-3637-4ad1-bbc7-59b74b3ea01e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81b90c27ef274539895af6a0918cc0dd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:28.813586 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1174/2375] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:59:28 2022] DELETE /identity/v3/users/81b90c27ef274539895af6a0918cc0dd => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:28.818863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f72b9-50ce-4d6f-8256-7c8b176b29cc None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:28.820010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f72b9-50ce-4d6f-8256-7c8b176b29cc None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:28.820461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f72b9-50ce-4d6f-8256-7c8b176b29cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:28.843140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-1d4f72b9-50ce-4d6f-8256-7c8b176b29cc None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:28.844537 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1d4f72b9-50ce-4d6f-8256-7c8b176b29cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:28.907897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1202/2376] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:59:28.919228 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c3089-3637-4ad1-bbc7-59b74b3ea01e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:28.920795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-098c3089-3637-4ad1-bbc7-59b74b3ea01e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:29.015077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:29.033391 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c3089-3637-4ad1-bbc7-59b74b3ea01e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:29.041526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57468bc-ab76-4542-af5f-84bca8f48af7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:29.044285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57468bc-ab76-4542-af5f-84bca8f48af7 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:29.044285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57468bc-ab76-4542-af5f-84bca8f48af7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:29.044285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57468bc-ab76-4542-af5f-84bca8f48af7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:29.058267 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57468bc-ab76-4542-af5f-84bca8f48af7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:29.059766 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57468bc-ab76-4542-af5f-84bca8f48af7 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:29.072117 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1175/2377] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:29.628631 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f72b9-50ce-4d6f-8256-7c8b176b29cc None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:29.630320 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1d4f72b9-50ce-4d6f-8256-7c8b176b29cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:29.647566 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f72b9-50ce-4d6f-8256-7c8b176b29cc None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:29.653015 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cbfdb-8c47-44b5-a1af-c5ebd147a6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:29.654076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cbfdb-8c47-44b5-a1af-c5ebd147a6d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:29.654626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cbfdb-8c47-44b5-a1af-c5ebd147a6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:29.655006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cbfdb-8c47-44b5-a1af-c5ebd147a6d2 admin admin] PATH_INFO: `/v3/projects/1c701547b70343aea7503b7d7b05ecb7` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:29.656373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cbfdb-8c47-44b5-a1af-c5ebd147a6d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c701547b70343aea7503b7d7b05ecb7)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:29.658516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cbfdb-8c47-44b5-a1af-c5ebd147a6d2 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:29.781679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-2c7cbfdb-8c47-44b5-a1af-c5ebd147a6d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c701547b70343aea7503b7d7b05ecb7', 'request_id': 'req-2c7cbfdb-8c47-44b5-a1af-c5ebd147a6d2'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:29.788166 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-2c7cbfdb-8c47-44b5-a1af-c5ebd147a6d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c701547b70343aea7503b7d7b05ecb7', 'request_id': 'req-2c7cbfdb-8c47-44b5-a1af-c5ebd147a6d2'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:29.848197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-2c7cbfdb-8c47-44b5-a1af-c5ebd147a6d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c701547b70343aea7503b7d7b05ecb7', 'request_id': 'req-2c7cbfdb-8c47-44b5-a1af-c5ebd147a6d2'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:29.862695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-2c7cbfdb-8c47-44b5-a1af-c5ebd147a6d2 admin admin] The token cache is being invalidate because project 1c701547b70343aea7503b7d7b05ecb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:59:29.862986 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-2c7cbfdb-8c47-44b5-a1af-c5ebd147a6d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:29.863685 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-2c7cbfdb-8c47-44b5-a1af-c5ebd147a6d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:29.865907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1203/2378] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:59:29 2022] DELETE /identity/v3/projects/1c701547b70343aea7503b7d7b05ecb7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:29.870771 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954daa7-7dfd-4db2-8a6b-7ff3ea690eef None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:29.871380 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954daa7-7dfd-4db2-8a6b-7ff3ea690eef None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:29.871380 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954daa7-7dfd-4db2-8a6b-7ff3ea690eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:29.893141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f954daa7-7dfd-4db2-8a6b-7ff3ea690eef None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:29.894356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f954daa7-7dfd-4db2-8a6b-7ff3ea690eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:29.968568 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1176/2379] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:29.977262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a35c8-8743-44e3-8eb2-caf9c7b01808 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:29.977728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a35c8-8743-44e3-8eb2-caf9c7b01808 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:29.978015 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a35c8-8743-44e3-8eb2-caf9c7b01808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:29.989510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954daa7-7dfd-4db2-8a6b-7ff3ea690eef None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:29.991643 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f954daa7-7dfd-4db2-8a6b-7ff3ea690eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:30.000224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-fb0a35c8-8743-44e3-8eb2-caf9c7b01808 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:30.001462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-fb0a35c8-8743-44e3-8eb2-caf9c7b01808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:30.006239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954daa7-7dfd-4db2-8a6b-7ff3ea690eef None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:30.011191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275eb9c-4b23-4584-9c98-8a55e4e5a8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:30.011909 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275eb9c-4b23-4584-9c98-8a55e4e5a8e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:30.012256 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275eb9c-4b23-4584-9c98-8a55e4e5a8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:30.012605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275eb9c-4b23-4584-9c98-8a55e4e5a8e2 admin admin] PATH_INFO: `/v3/users/664c4d177c8349b6825b9f7fb738c5af` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:30.013950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275eb9c-4b23-4584-9c98-8a55e4e5a8e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=664c4d177c8349b6825b9f7fb738c5af)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:30.015682 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1204/2380] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:30.016357 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275eb9c-4b23-4584-9c98-8a55e4e5a8e2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:30.016468 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a35c8-8743-44e3-8eb2-caf9c7b01808 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:30.025495 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a35c8-8743-44e3-8eb2-caf9c7b01808 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:30.030256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488bc5b-ee45-4e04-a9f5-bb6d31cce890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:30.030848 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488bc5b-ee45-4e04-a9f5-bb6d31cce890 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:30.031151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488bc5b-ee45-4e04-a9f5-bb6d31cce890 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:30.034379 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488bc5b-ee45-4e04-a9f5-bb6d31cce890 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:30.034379 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488bc5b-ee45-4e04-a9f5-bb6d31cce890 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:59:30.034379 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488bc5b-ee45-4e04-a9f5-bb6d31cce890 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:30.034379 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488bc5b-ee45-4e04-a9f5-bb6d31cce890 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:30.046864 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1205/2381] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:59:30.051109 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488bc5b-ee45-4e04-a9f5-bb6d31cce890 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:30.061191 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488bc5b-ee45-4e04-a9f5-bb6d31cce890 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:30.065977 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe614dd-8277-4620-918f-8d9426f01e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:30.066541 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe614dd-8277-4620-918f-8d9426f01e4f admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:30.066816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe614dd-8277-4620-918f-8d9426f01e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:30.067085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe614dd-8277-4620-918f-8d9426f01e4f admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:30.067725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe614dd-8277-4620-918f-8d9426f01e4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:30.070500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe614dd-8277-4620-918f-8d9426f01e4f admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:30.163215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1206/2382] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:59:30 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:59:30.164740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5275eb9c-4b23-4584-9c98-8a55e4e5a8e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '664c4d177c8349b6825b9f7fb738c5af'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:30.165100 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe614dd-8277-4620-918f-8d9426f01e4f admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:30.172826 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5275eb9c-4b23-4584-9c98-8a55e4e5a8e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '664c4d177c8349b6825b9f7fb738c5af'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:30.173613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5275eb9c-4b23-4584-9c98-8a55e4e5a8e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '664c4d177c8349b6825b9f7fb738c5af'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:30.184126 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:30.229544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe614dd-8277-4620-918f-8d9426f01e4f admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:30.235610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86d798-b151-4d48-8d20-ee4aa3a2a5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:30.236371 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86d798-b151-4d48-8d20-ee4aa3a2a5ef admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:30.236609 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86d798-b151-4d48-8d20-ee4aa3a2a5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:30.236951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86d798-b151-4d48-8d20-ee4aa3a2a5ef admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:30.237914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86d798-b151-4d48-8d20-ee4aa3a2a5ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:30.239452 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5275eb9c-4b23-4584-9c98-8a55e4e5a8e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '664c4d177c8349b6825b9f7fb738c5af'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:30.239741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86d798-b151-4d48-8d20-ee4aa3a2a5ef admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:30.270063 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5275eb9c-4b23-4584-9c98-8a55e4e5a8e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '664c4d177c8349b6825b9f7fb738c5af'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:30.277787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1177/2383] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 14:59:29 2022] DELETE /identity/v3/users/664c4d177c8349b6825b9f7fb738c5af => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:59:30.282430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f41a0-2512-43d6-938c-00f0616588af None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:30.282720 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f41a0-2512-43d6-938c-00f0616588af None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:30.283085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f41a0-2512-43d6-938c-00f0616588af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:30.302769 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ef6f41a0-2512-43d6-938c-00f0616588af None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:30.304017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ef6f41a0-2512-43d6-938c-00f0616588af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:30.327167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1207/2384] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:59:30 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:59:30.330821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86d798-b151-4d48-8d20-ee4aa3a2a5ef admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:30.344971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-3b86d798-b151-4d48-8d20-ee4aa3a2a5ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:30.367866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86d798-b151-4d48-8d20-ee4aa3a2a5ef admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:30.373965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:30.386067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1178/2385] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:30.393577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f41a0-2512-43d6-938c-00f0616588af None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:30.395660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ef6f41a0-2512-43d6-938c-00f0616588af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:30.472153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e040e-4017-43fd-a44b-692562026a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:30.473215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e040e-4017-43fd-a44b-692562026a78 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:30.473437 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e040e-4017-43fd-a44b-692562026a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:30.473706 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e040e-4017-43fd-a44b-692562026a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:30.475256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e040e-4017-43fd-a44b-692562026a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:30.483366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e040e-4017-43fd-a44b-692562026a78 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:30.491632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f41a0-2512-43d6-938c-00f0616588af None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:30.495000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202a271-c6b8-49b2-a359-b20a5d4d47d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:30.495897 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202a271-c6b8-49b2-a359-b20a5d4d47d5 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:30.496280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202a271-c6b8-49b2-a359-b20a5d4d47d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:30.496709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202a271-c6b8-49b2-a359-b20a5d4d47d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:30.500210 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1208/2386] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:30.503698 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e040e-4017-43fd-a44b-692562026a78 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:30.513411 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202a271-c6b8-49b2-a359-b20a5d4d47d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:30.517775 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e040e-4017-43fd-a44b-692562026a78 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:30.517909 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202a271-c6b8-49b2-a359-b20a5d4d47d5 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:30.527371 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f91d55-8acb-495b-92c7-defb1c732715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:30.528413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f91d55-8acb-495b-92c7-defb1c732715 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:30.528947 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1179/2387] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:30.529158 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f91d55-8acb-495b-92c7-defb1c732715 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:30.529525 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f91d55-8acb-495b-92c7-defb1c732715 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:30.530317 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f91d55-8acb-495b-92c7-defb1c732715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:30.532090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f91d55-8acb-495b-92c7-defb1c732715 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:30.547740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1209/2388] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:30.551729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202a271-c6b8-49b2-a359-b20a5d4d47d5 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:30.562790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202a271-c6b8-49b2-a359-b20a5d4d47d5 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:30.575242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729db44-7160-425c-bd40-5d89804d8b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:30.577221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729db44-7160-425c-bd40-5d89804d8b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:30.577221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729db44-7160-425c-bd40-5d89804d8b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:30.577669 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729db44-7160-425c-bd40-5d89804d8b49 admin admin] PATH_INFO: `/v3/projects/557a5dfba4324952bcdf72f58da8a0dc/users/d7dd8c6c2ba8426383bbb86b2969d43a/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:30.601528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729db44-7160-425c-bd40-5d89804d8b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=557a5dfba4324952bcdf72f58da8a0dc, user_id=d7dd8c6c2ba8426383bbb86b2969d43a, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:30.604947 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729db44-7160-425c-bd40-5d89804d8b49 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:30.747052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1180/2389] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:30 2022] PUT /identity/v3/projects/557a5dfba4324952bcdf72f58da8a0dc/users/d7dd8c6c2ba8426383bbb86b2969d43a/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:59:30.747400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b00dc-6c3d-4fb8-a900-c2d4a3caf94e None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:30.747400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b00dc-6c3d-4fb8-a900-c2d4a3caf94e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:30.747400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b00dc-6c3d-4fb8-a900-c2d4a3caf94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:30.754566 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-255b00dc-6c3d-4fb8-a900-c2d4a3caf94e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:30.756966 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-255b00dc-6c3d-4fb8-a900-c2d4a3caf94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:30.825866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1210/2390] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:59:30.836313 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729db44-7160-425c-bd40-5d89804d8b49 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:30.910061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729db44-7160-425c-bd40-5d89804d8b49 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:30.914038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4b89c-897d-4bbc-a85e-0d4859a7fade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:30.914685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4b89c-897d-4bbc-a85e-0d4859a7fade service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:30.914978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4b89c-897d-4bbc-a85e-0d4859a7fade service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:30.915320 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4b89c-897d-4bbc-a85e-0d4859a7fade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:30.925188 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4b89c-897d-4bbc-a85e-0d4859a7fade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:30.926695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4b89c-897d-4bbc-a85e-0d4859a7fade service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:30.945061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1181/2391] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:31.497712 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b00dc-6c3d-4fb8-a900-c2d4a3caf94e None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:31.499314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-255b00dc-6c3d-4fb8-a900-c2d4a3caf94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:31.517072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b00dc-6c3d-4fb8-a900-c2d4a3caf94e None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:31.530439 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ce367-0ab6-4724-9221-4607dbbfc7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:31.531704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ce367-0ab6-4724-9221-4607dbbfc7c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:31.532296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ce367-0ab6-4724-9221-4607dbbfc7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:31.532871 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ce367-0ab6-4724-9221-4607dbbfc7c9 admin admin] PATH_INFO: `/v3/projects/3bd055911f224d00af5f07debb5a6d46` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:31.535050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ce367-0ab6-4724-9221-4607dbbfc7c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd055911f224d00af5f07debb5a6d46)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:31.537786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ce367-0ab6-4724-9221-4607dbbfc7c9 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:31.691815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:31.706414 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-666ce367-0ab6-4724-9221-4607dbbfc7c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd055911f224d00af5f07debb5a6d46', 'request_id': 'req-666ce367-0ab6-4724-9221-4607dbbfc7c9'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:31.713840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-666ce367-0ab6-4724-9221-4607dbbfc7c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd055911f224d00af5f07debb5a6d46', 'request_id': 'req-666ce367-0ab6-4724-9221-4607dbbfc7c9'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:32.181393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-666ce367-0ab6-4724-9221-4607dbbfc7c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd055911f224d00af5f07debb5a6d46', 'request_id': 'req-666ce367-0ab6-4724-9221-4607dbbfc7c9'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:32.193702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-666ce367-0ab6-4724-9221-4607dbbfc7c9 admin admin] The token cache is being invalidate because project 3bd055911f224d00af5f07debb5a6d46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:59:32.194014 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-666ce367-0ab6-4724-9221-4607dbbfc7c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:32.194648 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-666ce367-0ab6-4724-9221-4607dbbfc7c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:59:32.196451 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1211/2392] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 14:59:31 2022] DELETE /identity/v3/projects/3bd055911f224d00af5f07debb5a6d46 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:32.201503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00626531-d408-48ca-9d5f-7363caee990f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:32.202366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00626531-d408-48ca-9d5f-7363caee990f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:32.202803 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00626531-d408-48ca-9d5f-7363caee990f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:32.231158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-00626531-d408-48ca-9d5f-7363caee990f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:32.231158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-00626531-d408-48ca-9d5f-7363caee990f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:32.297901 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1182/2393] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:32.306503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cfe89e-56d4-4980-8bb8-ffb84c7b0725 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:32.306730 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cfe89e-56d4-4980-8bb8-ffb84c7b0725 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:32.307005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cfe89e-56d4-4980-8bb8-ffb84c7b0725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:32.326459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-60cfe89e-56d4-4980-8bb8-ffb84c7b0725 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:32.327687 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-60cfe89e-56d4-4980-8bb8-ffb84c7b0725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:32.335662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1212/2394] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:32.339981 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00626531-d408-48ca-9d5f-7363caee990f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:32.350106 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00626531-d408-48ca-9d5f-7363caee990f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:32.355012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef4514-c69e-46a5-9dd3-7bc3b9604fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:32.355639 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef4514-c69e-46a5-9dd3-7bc3b9604fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:32.355918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef4514-c69e-46a5-9dd3-7bc3b9604fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:32.356189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef4514-c69e-46a5-9dd3-7bc3b9604fe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:32.357012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef4514-c69e-46a5-9dd3-7bc3b9604fe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:59:32.357185 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef4514-c69e-46a5-9dd3-7bc3b9604fe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:32.358849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef4514-c69e-46a5-9dd3-7bc3b9604fe7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:32.374505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1183/2395] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 14:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:59:32.376011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cfe89e-56d4-4980-8bb8-ffb84c7b0725 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:32.385026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cfe89e-56d4-4980-8bb8-ffb84c7b0725 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:32.390223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a498f24-a2cb-47f5-a7b7-0185a02963aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:32.391249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a498f24-a2cb-47f5-a7b7-0185a02963aa admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:32.391249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a498f24-a2cb-47f5-a7b7-0185a02963aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:32.391463 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a498f24-a2cb-47f5-a7b7-0185a02963aa admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:32.392110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a498f24-a2cb-47f5-a7b7-0185a02963aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:32.394126 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a498f24-a2cb-47f5-a7b7-0185a02963aa admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:32.532401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1213/2396] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:59:32 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:32.537754 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef4514-c69e-46a5-9dd3-7bc3b9604fe7 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:32.547734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef4514-c69e-46a5-9dd3-7bc3b9604fe7 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:32.552278 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:32.602777 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3d1eb-ae8c-4750-bae2-be5cf8362bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:32.603561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3d1eb-ae8c-4750-bae2-be5cf8362bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:32.603841 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3d1eb-ae8c-4750-bae2-be5cf8362bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:32.604116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3d1eb-ae8c-4750-bae2-be5cf8362bb8 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:32.604762 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3d1eb-ae8c-4750-bae2-be5cf8362bb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:32.606689 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3d1eb-ae8c-4750-bae2-be5cf8362bb8 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:32.730763 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1184/2397] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:59:32 2022] POST /identity/v3/users => generated 290 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:59:32.734895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a498f24-a2cb-47f5-a7b7-0185a02963aa admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:32.744993 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a498f24-a2cb-47f5-a7b7-0185a02963aa admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:32.749450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d88218-8cda-4493-a035-52890127d902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:32.750038 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d88218-8cda-4493-a035-52890127d902 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:32.750316 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d88218-8cda-4493-a035-52890127d902 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:32.750587 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d88218-8cda-4493-a035-52890127d902 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:32.751354 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d88218-8cda-4493-a035-52890127d902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:32.752945 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d88218-8cda-4493-a035-52890127d902 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:32.765930 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1214/2398] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:32.769980 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3d1eb-ae8c-4750-bae2-be5cf8362bb8 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:32.780539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3d1eb-ae8c-4750-bae2-be5cf8362bb8 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:32.785159 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cbad9-3cb2-497d-8fa7-632ce25154ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:32.785757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cbad9-3cb2-497d-8fa7-632ce25154ea admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:32.786036 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cbad9-3cb2-497d-8fa7-632ce25154ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:32.786331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cbad9-3cb2-497d-8fa7-632ce25154ea admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:32.787638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527cbad9-3cb2-497d-8fa7-632ce25154ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:32.789228 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527cbad9-3cb2-497d-8fa7-632ce25154ea admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:32.799708 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1185/2399] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:32.803824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d88218-8cda-4493-a035-52890127d902 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:32.813673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d88218-8cda-4493-a035-52890127d902 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:32.818762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad40108-b525-4115-84a8-633723882929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:32.819463 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad40108-b525-4115-84a8-633723882929 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:32.819746 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad40108-b525-4115-84a8-633723882929 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:32.820017 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad40108-b525-4115-84a8-633723882929 admin admin] PATH_INFO: `/v3/projects/f4a168c27895481dbc738788feefa2a8/users/c323addb0b2d45ffb8568d6ebfb4a9eb/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:32.834895 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad40108-b525-4115-84a8-633723882929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4a168c27895481dbc738788feefa2a8, user_id=c323addb0b2d45ffb8568d6ebfb4a9eb, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:32.837664 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad40108-b525-4115-84a8-633723882929 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:32.918293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1215/2400] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:32 2022] PUT /identity/v3/projects/f4a168c27895481dbc738788feefa2a8/users/c323addb0b2d45ffb8568d6ebfb4a9eb/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:32.923529 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4190e73-f150-4c44-9e0b-5c7ed642acd7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:32.923818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4190e73-f150-4c44-9e0b-5c7ed642acd7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:32.924090 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4190e73-f150-4c44-9e0b-5c7ed642acd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:32.943519 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f4190e73-f150-4c44-9e0b-5c7ed642acd7 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:32.944860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f4190e73-f150-4c44-9e0b-5c7ed642acd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:32.989883 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbed75-2dc6-466b-a01e-62b13138e90a None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:32.990156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbed75-2dc6-466b-a01e-62b13138e90a None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:32.990833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbed75-2dc6-466b-a01e-62b13138e90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:33.015732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1186/2401] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:33.019815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-0cfbed75-2dc6-466b-a01e-62b13138e90a None None] MFA Rules not processed for user `d7dd8c6c2ba8426383bbb86b2969d43a`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:33.020877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0cfbed75-2dc6-466b-a01e-62b13138e90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:33.027720 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4190e73-f150-4c44-9e0b-5c7ed642acd7 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:33.029217 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f4190e73-f150-4c44-9e0b-5c7ed642acd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:33.095118 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4190e73-f150-4c44-9e0b-5c7ed642acd7 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:33.098828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1216/2402] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:59:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:59:33.103688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9700ce4-240a-466f-93e5-08ea4cd98896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:33.104256 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9700ce4-240a-466f-93e5-08ea4cd98896 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:33.104602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9700ce4-240a-466f-93e5-08ea4cd98896 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:33.104936 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9700ce4-240a-466f-93e5-08ea4cd98896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:33.106775 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbed75-2dc6-466b-a01e-62b13138e90a None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:33.108706 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0cfbed75-2dc6-466b-a01e-62b13138e90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:33.120316 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9700ce4-240a-466f-93e5-08ea4cd98896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:33.120316 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9700ce4-240a-466f-93e5-08ea4cd98896 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:33.129291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1187/2403] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:59:33.163885 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:33.178647 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbed75-2dc6-466b-a01e-62b13138e90a None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:33.182498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab53f15-185a-42b4-a226-35367624a82a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:33.183122 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab53f15-185a-42b4-a226-35367624a82a service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:33.183410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab53f15-185a-42b4-a226-35367624a82a service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:33.183836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab53f15-185a-42b4-a226-35367624a82a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:33.192492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab53f15-185a-42b4-a226-35367624a82a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:33.194576 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab53f15-185a-42b4-a226-35367624a82a service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:33.203531 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1217/2404] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:34.963382 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d959019-fe61-4c43-adb5-53a5b1b5a853 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:34.969540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d959019-fe61-4c43-adb5-53a5b1b5a853 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:34.969540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d959019-fe61-4c43-adb5-53a5b1b5a853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:34.984033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-9d959019-fe61-4c43-adb5-53a5b1b5a853 None None] MFA Rules not processed for user `d7dd8c6c2ba8426383bbb86b2969d43a`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:34.985249 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9d959019-fe61-4c43-adb5-53a5b1b5a853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:34.992553 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1188/2405] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:59:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:39.377255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab53f15-185a-42b4-a226-35367624a82a service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:39.378804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-5ab53f15-185a-42b4-a226-35367624a82a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:39.474949 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab53f15-185a-42b4-a226-35367624a82a service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:39.483470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734e07a-dc71-4c1c-a4ef-124c66deb39b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4f11655cd51438e9652869f74ffa975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:39.484087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734e07a-dc71-4c1c-a4ef-124c66deb39b service placement] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:39.484361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734e07a-dc71-4c1c-a4ef-124c66deb39b service placement] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:39.484631 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734e07a-dc71-4c1c-a4ef-124c66deb39b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:39.486375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6734e07a-dc71-4c1c-a4ef-124c66deb39b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:39.499004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6734e07a-dc71-4c1c-a4ef-124c66deb39b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:39.500729 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6734e07a-dc71-4c1c-a4ef-124c66deb39b service placement] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:39.513499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1218/2406] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:59:39.606186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5ba81-9813-4efd-bd23-64a865803ff2 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:39.606672 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5ba81-9813-4efd-bd23-64a865803ff2 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:39.606782 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5ba81-9813-4efd-bd23-64a865803ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:39.636758 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-89a5ba81-9813-4efd-bd23-64a865803ff2 None None] MFA Rules not processed for user `c323addb0b2d45ffb8568d6ebfb4a9eb`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:39.638703 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-89a5ba81-9813-4efd-bd23-64a865803ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:39.652253 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:39.734597 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1189/2407] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:59:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:39.736299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734e07a-dc71-4c1c-a4ef-124c66deb39b service placement] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:39.747775 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734e07a-dc71-4c1c-a4ef-124c66deb39b service placement] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:39.753989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5db0d-fbee-4b58-bc39-476ab35ba8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:39.754692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5db0d-fbee-4b58-bc39-476ab35ba8c8 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:39.755761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5db0d-fbee-4b58-bc39-476ab35ba8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:39.756038 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5db0d-fbee-4b58-bc39-476ab35ba8c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:39.756687 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5db0d-fbee-4b58-bc39-476ab35ba8c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:39.758618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5db0d-fbee-4b58-bc39-476ab35ba8c8 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:39.852791 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1219/2408] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 14:59:39 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:39.857078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5ba81-9813-4efd-bd23-64a865803ff2 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:39.868334 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5ba81-9813-4efd-bd23-64a865803ff2 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:39.934712 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cd0ca-78a2-4340-b3b6-711615b2d32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:39.935585 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cd0ca-78a2-4340-b3b6-711615b2d32c admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:39.936047 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cd0ca-78a2-4340-b3b6-711615b2d32c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:39.936473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cd0ca-78a2-4340-b3b6-711615b2d32c admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:39.937467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57cd0ca-78a2-4340-b3b6-711615b2d32c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:39.940808 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57cd0ca-78a2-4340-b3b6-711615b2d32c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:40.049801 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1190/2409] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 14:59:39 2022] POST /identity/v3/users => generated 295 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:59:40.054498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5db0d-fbee-4b58-bc39-476ab35ba8c8 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:40.066019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5db0d-fbee-4b58-bc39-476ab35ba8c8 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:40.072691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783dff1-238b-4786-aa10-fa4d84f3b4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:40.073017 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783dff1-238b-4786-aa10-fa4d84f3b4cb admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:40.073535 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783dff1-238b-4786-aa10-fa4d84f3b4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:40.073676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783dff1-238b-4786-aa10-fa4d84f3b4cb admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:40.074450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783dff1-238b-4786-aa10-fa4d84f3b4cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:40.076214 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783dff1-238b-4786-aa10-fa4d84f3b4cb admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:40.109401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1220/2410] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:40.113680 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cd0ca-78a2-4340-b3b6-711615b2d32c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:40.130742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cd0ca-78a2-4340-b3b6-711615b2d32c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:40.136682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca25639-6aba-4548-b75a-d0a0713e690c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:40.137474 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca25639-6aba-4548-b75a-d0a0713e690c admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:40.137824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca25639-6aba-4548-b75a-d0a0713e690c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:40.138101 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca25639-6aba-4548-b75a-d0a0713e690c admin admin] PATH_INFO: `/v3/projects/21821c00ee6c4b64ac3fa0cf81ef15d6/users/e11a8cab2d4e4db99f0e29ae8831b87b/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:40.154580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca25639-6aba-4548-b75a-d0a0713e690c admin admin] RBAC: Authorizing `identity:create_grant(project_id=21821c00ee6c4b64ac3fa0cf81ef15d6, user_id=e11a8cab2d4e4db99f0e29ae8831b87b, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:40.157403 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca25639-6aba-4548-b75a-d0a0713e690c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:40.282692 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1191/2411] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:40 2022] PUT /identity/v3/projects/21821c00ee6c4b64ac3fa0cf81ef15d6/users/e11a8cab2d4e4db99f0e29ae8831b87b/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:40.283473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783dff1-238b-4786-aa10-fa4d84f3b4cb admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:40.283473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783dff1-238b-4786-aa10-fa4d84f3b4cb admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:40.342328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e2de4-c45f-4f4a-b97f-fb28fb392957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:40.342907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e2de4-c45f-4f4a-b97f-fb28fb392957 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:40.344202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e2de4-c45f-4f4a-b97f-fb28fb392957 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:40.344202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e2de4-c45f-4f4a-b97f-fb28fb392957 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:40.344202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e2de4-c45f-4f4a-b97f-fb28fb392957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:40.345695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e2de4-c45f-4f4a-b97f-fb28fb392957 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:40.357954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1221/2412] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 14:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:59:40.362209 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca25639-6aba-4548-b75a-d0a0713e690c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:40.374445 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca25639-6aba-4548-b75a-d0a0713e690c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:40.380733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5706812-6cec-49d4-8d42-172760da0c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:40.381512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5706812-6cec-49d4-8d42-172760da0c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:40.382234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5706812-6cec-49d4-8d42-172760da0c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:40.382234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5706812-6cec-49d4-8d42-172760da0c04 admin admin] PATH_INFO: `/v3/projects/21821c00ee6c4b64ac3fa0cf81ef15d6/users/e11a8cab2d4e4db99f0e29ae8831b87b/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:40.385881 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5706812-6cec-49d4-8d42-172760da0c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21821c00ee6c4b64ac3fa0cf81ef15d6, user_id=e11a8cab2d4e4db99f0e29ae8831b87b, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:40.388738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5706812-6cec-49d4-8d42-172760da0c04 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:40.415606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1192/2413] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 14:59:40 2022] PUT /identity/v3/projects/21821c00ee6c4b64ac3fa0cf81ef15d6/users/e11a8cab2d4e4db99f0e29ae8831b87b/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:59:46.670410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4f111-df84-4264-82fe-fb57bc20c831 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:46.670978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4f111-df84-4264-82fe-fb57bc20c831 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:46.671209 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4f111-df84-4264-82fe-fb57bc20c831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:46.693998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-2cf4f111-df84-4264-82fe-fb57bc20c831 None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:46.695257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-2cf4f111-df84-4264-82fe-fb57bc20c831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:46.729011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:46.763526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1222/2414] 10.222.0.27 () {58 vars in 967 bytes} [Tue Aug 30 14:59:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:46.780722 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5706812-6cec-49d4-8d42-172760da0c04 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:46.783988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d5706812-6cec-49d4-8d42-172760da0c04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:46.909181 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5706812-6cec-49d4-8d42-172760da0c04 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:46.915713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d282ba1-5e26-46a4-8965-2a0c29c7520b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:46.916645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d282ba1-5e26-46a4-8965-2a0c29c7520b service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:46.917098 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d282ba1-5e26-46a4-8965-2a0c29c7520b service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:46.917531 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d282ba1-5e26-46a4-8965-2a0c29c7520b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:46.925596 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:46.928247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d282ba1-5e26-46a4-8965-2a0c29c7520b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:46.929984 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d282ba1-5e26-46a4-8965-2a0c29c7520b service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:46.939140 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1193/2415] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 14:59:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:59:47.456654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161a721-e4eb-4bd6-ba1b-72d525e1bff5 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:47.456961 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161a721-e4eb-4bd6-ba1b-72d525e1bff5 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:47.457219 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161a721-e4eb-4bd6-ba1b-72d525e1bff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:47.508975 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e161a721-e4eb-4bd6-ba1b-72d525e1bff5 None None] MFA Rules not processed for user `e11a8cab2d4e4db99f0e29ae8831b87b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:59:47.508975 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e161a721-e4eb-4bd6-ba1b-72d525e1bff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:47.582658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1223/2416] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 14:59:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:59:47.596907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d282ba1-5e26-46a4-8965-2a0c29c7520b service swift] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:47.598562 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6d282ba1-5e26-46a4-8965-2a0c29c7520b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:47.677760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d282ba1-5e26-46a4-8965-2a0c29c7520b service swift] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:47.683940 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb4af1-8892-42e5-b63b-cf7bdbcd5cfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:47.687851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb4af1-8892-42e5-b63b-cf7bdbcd5cfc service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:47.687851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb4af1-8892-42e5-b63b-cf7bdbcd5cfc service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:47.687851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb4af1-8892-42e5-b63b-cf7bdbcd5cfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:47.752218 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb4af1-8892-42e5-b63b-cf7bdbcd5cfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:47.753822 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb4af1-8892-42e5-b63b-cf7bdbcd5cfc service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:47.766755 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1194/2417] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:57.579202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=43 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:57.762451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=58 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:00:13.437702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ccd10-a436-493c-b7d7-5ef2830510db None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:00:13.438220 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ccd10-a436-493c-b7d7-5ef2830510db None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:00:13.438314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ccd10-a436-493c-b7d7-5ef2830510db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:00:13.457387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-088ccd10-a436-493c-b7d7-5ef2830510db None None] MFA Rules not processed for user `c323addb0b2d45ffb8568d6ebfb4a9eb`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:00:13.458584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-088ccd10-a436-493c-b7d7-5ef2830510db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:00:13.466317 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1224/2418] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:00:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:00:23.452631 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:00:28.346636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe6b5b-ff6e-47ad-95be-503c2ef5a3ec None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:00:28.347164 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe6b5b-ff6e-47ad-95be-503c2ef5a3ec None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:00:28.347263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe6b5b-ff6e-47ad-95be-503c2ef5a3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:00:28.366738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-63fe6b5b-ff6e-47ad-95be-503c2ef5a3ec None None] MFA Rules not processed for user `39d98294ba884280b4971af36a9fbd6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:00:28.367951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-63fe6b5b-ff6e-47ad-95be-503c2ef5a3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:00:28.425516 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1195/2419] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:00:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:00:38.418785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=25 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:00:45.523270 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ccd10-a436-493c-b7d7-5ef2830510db None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:00:45.524628 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-088ccd10-a436-493c-b7d7-5ef2830510db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:00:45.582263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ccd10-a436-493c-b7d7-5ef2830510db None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:00:45.585857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cdb0c-ef9e-4899-afa1-c5295d4066ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:00:45.586458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cdb0c-ef9e-4899-afa1-c5295d4066ad service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:00:45.586741 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cdb0c-ef9e-4899-afa1-c5295d4066ad service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:00:45.587011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cdb0c-ef9e-4899-afa1-c5295d4066ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:00:45.596645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58cdb0c-ef9e-4899-afa1-c5295d4066ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:00:45.598235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58cdb0c-ef9e-4899-afa1-c5295d4066ad service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:00:45.608703 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1225/2420] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:00:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:00:54.532161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a2e1f-95d4-437c-834a-ebf9f628942d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:00:54.532637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a2e1f-95d4-437c-834a-ebf9f628942d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:00:54.532841 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a2e1f-95d4-437c-834a-ebf9f628942d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:00:54.552303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2e8a2e1f-95d4-437c-834a-ebf9f628942d None None] MFA Rules not processed for user `7dc5cd44b1b34a7ebbdca2c062715431`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:00:54.555718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2e8a2e1f-95d4-437c-834a-ebf9f628942d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:00:54.611130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1196/2421] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:00:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:00:55.603768 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=26 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:00:59.168268 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25d7e5-e28c-4d75-9990-8ecc3f504fd8 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:00:59.168851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25d7e5-e28c-4d75-9990-8ecc3f504fd8 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:00:59.168851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25d7e5-e28c-4d75-9990-8ecc3f504fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:00:59.188165 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-fc25d7e5-e28c-4d75-9990-8ecc3f504fd8 None None] MFA Rules not processed for user `d7dd8c6c2ba8426383bbb86b2969d43a`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:00:59.189293 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-fc25d7e5-e28c-4d75-9990-8ecc3f504fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:00:59.243204 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1226/2422] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:00:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:01:04.604443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=25 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:01:09.236459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=25 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:01:19.474437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80bf5c-b643-4de0-8c28-8f0ef1d4cc0c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:19.474926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80bf5c-b643-4de0-8c28-8f0ef1d4cc0c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:19.475334 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80bf5c-b643-4de0-8c28-8f0ef1d4cc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:19.493866 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-8e80bf5c-b643-4de0-8c28-8f0ef1d4cc0c None None] MFA Rules not processed for user `3bd31b8807fc450f8f134417cd535542`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:01:19.495046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8e80bf5c-b643-4de0-8c28-8f0ef1d4cc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:01:19.547314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1197/2423] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:01:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:01:29.540538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=25 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:01:30.146169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25d7e5-e28c-4d75-9990-8ecc3f504fd8 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:01:30.155775 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25d7e5-e28c-4d75-9990-8ecc3f504fd8 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:01:30.159625 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8408d-0242-4ef5-b98c-226298c39a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:01:30.160326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8408d-0242-4ef5-b98c-226298c39a85 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:30.160603 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8408d-0242-4ef5-b98c-226298c39a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:30.160866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8408d-0242-4ef5-b98c-226298c39a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:30.162651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0db8408d-0242-4ef5-b98c-226298c39a85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:01:30.176559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8408d-0242-4ef5-b98c-226298c39a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:01:30.177990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8408d-0242-4ef5-b98c-226298c39a85 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:01:30.189241 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1227/2424] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:01:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:01:32.870083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80bf5c-b643-4de0-8c28-8f0ef1d4cc0c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:01:32.879906 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80bf5c-b643-4de0-8c28-8f0ef1d4cc0c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:01:32.883623 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191a313-da18-4732-a345-3f2e97b6fad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:01:32.884227 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191a313-da18-4732-a345-3f2e97b6fad6 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:32.884508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191a313-da18-4732-a345-3f2e97b6fad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:32.884779 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191a313-da18-4732-a345-3f2e97b6fad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:32.886539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e191a313-da18-4732-a345-3f2e97b6fad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:01:32.899889 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191a313-da18-4732-a345-3f2e97b6fad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:01:32.901305 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191a313-da18-4732-a345-3f2e97b6fad6 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:01:32.912430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1198/2425] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:01:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:01:40.184082 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=6 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:01:42.907269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:01:45.008241 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8408d-0242-4ef5-b98c-226298c39a85 service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:01:45.019728 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8408d-0242-4ef5-b98c-226298c39a85 service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:01:45.025404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27dab99-1870-4e79-9281-57598f5072b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:01:45.026098 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27dab99-1870-4e79-9281-57598f5072b2 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:45.026373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27dab99-1870-4e79-9281-57598f5072b2 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:45.026642 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27dab99-1870-4e79-9281-57598f5072b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:45.028464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e27dab99-1870-4e79-9281-57598f5072b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:01:45.040210 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27dab99-1870-4e79-9281-57598f5072b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:01:45.041922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27dab99-1870-4e79-9281-57598f5072b2 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:01:45.052632 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1228/2426] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:01:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:01:54.701360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191a313-da18-4732-a345-3f2e97b6fad6 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:01:54.711321 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191a313-da18-4732-a345-3f2e97b6fad6 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:01:54.714982 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2cce6-a703-4d03-b357-a3d78f908778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:01:54.715698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2cce6-a703-4d03-b357-a3d78f908778 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:54.716048 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2cce6-a703-4d03-b357-a3d78f908778 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:54.716329 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2cce6-a703-4d03-b357-a3d78f908778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:54.718062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-33f2cce6-a703-4d03-b357-a3d78f908778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:01:54.791953 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f2cce6-a703-4d03-b357-a3d78f908778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:01:54.793455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f2cce6-a703-4d03-b357-a3d78f908778 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:01:54.803164 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1199/2427] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:01:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:01:55.046957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=6 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:01:58.750753 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27dab99-1870-4e79-9281-57598f5072b2 service nova] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:01:58.752358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e27dab99-1870-4e79-9281-57598f5072b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:01:58.770066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27dab99-1870-4e79-9281-57598f5072b2 service nova] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:01:58.771367 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0aaf1-41fc-48fe-b63a-8fc12597c87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:01:58.771991 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0aaf1-41fc-48fe-b63a-8fc12597c87d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:58.772272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0aaf1-41fc-48fe-b63a-8fc12597c87d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:58.772547 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0aaf1-41fc-48fe-b63a-8fc12597c87d admin admin] PATH_INFO: `/v3/users/39d98294ba884280b4971af36a9fbd6e` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:58.773700 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0aaf1-41fc-48fe-b63a-8fc12597c87d admin admin] RBAC: Authorizing `identity:delete_user(user_id=39d98294ba884280b4971af36a9fbd6e)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:01:58.775716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0aaf1-41fc-48fe-b63a-8fc12597c87d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:01:58.946016 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b6a0aaf1-41fc-48fe-b63a-8fc12597c87d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39d98294ba884280b4971af36a9fbd6e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:58.953122 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b6a0aaf1-41fc-48fe-b63a-8fc12597c87d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39d98294ba884280b4971af36a9fbd6e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:58.965525 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b6a0aaf1-41fc-48fe-b63a-8fc12597c87d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39d98294ba884280b4971af36a9fbd6e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:58.966207 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b6a0aaf1-41fc-48fe-b63a-8fc12597c87d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39d98294ba884280b4971af36a9fbd6e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:58.966807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b6a0aaf1-41fc-48fe-b63a-8fc12597c87d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39d98294ba884280b4971af36a9fbd6e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:59.063186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1229/2428] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:01:58 2022] DELETE /identity/v3/users/39d98294ba884280b4971af36a9fbd6e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:01:59.288200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2cce6-a703-4d03-b357-a3d78f908778 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:01:59.303028 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2cce6-a703-4d03-b357-a3d78f908778 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:01:59.358719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa151f-a2b0-43d0-9983-8ed4d6eb8a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:01:59.359389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa151f-a2b0-43d0-9983-8ed4d6eb8a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:59.360283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa151f-a2b0-43d0-9983-8ed4d6eb8a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:59.360283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa151f-a2b0-43d0-9983-8ed4d6eb8a2a admin admin] PATH_INFO: `/v3/projects/d33a78463eba4967826c9917dd352db2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:59.361169 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa151f-a2b0-43d0-9983-8ed4d6eb8a2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d33a78463eba4967826c9917dd352db2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:01:59.363160 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa151f-a2b0-43d0-9983-8ed4d6eb8a2a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:01:59.471788 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9efa151f-a2b0-43d0-9983-8ed4d6eb8a2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd33a78463eba4967826c9917dd352db2', 'request_id': 'req-9efa151f-a2b0-43d0-9983-8ed4d6eb8a2a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:59.478838 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9efa151f-a2b0-43d0-9983-8ed4d6eb8a2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd33a78463eba4967826c9917dd352db2', 'request_id': 'req-9efa151f-a2b0-43d0-9983-8ed4d6eb8a2a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:59.484357 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9efa151f-a2b0-43d0-9983-8ed4d6eb8a2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd33a78463eba4967826c9917dd352db2', 'request_id': 'req-9efa151f-a2b0-43d0-9983-8ed4d6eb8a2a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:59.568638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9efa151f-a2b0-43d0-9983-8ed4d6eb8a2a admin admin] The token cache is being invalidate because project d33a78463eba4967826c9917dd352db2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:01:59.568861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9efa151f-a2b0-43d0-9983-8ed4d6eb8a2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:59.569478 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9efa151f-a2b0-43d0-9983-8ed4d6eb8a2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:01:59.570964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1200/2429] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:01:59 2022] DELETE /identity/v3/projects/d33a78463eba4967826c9917dd352db2 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:01:59.576039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469acd5-3c62-4625-9f13-8d468d6b9959 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:59.576344 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469acd5-3c62-4625-9f13-8d468d6b9959 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:59.576615 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469acd5-3c62-4625-9f13-8d468d6b9959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:59.595854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-7469acd5-3c62-4625-9f13-8d468d6b9959 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:01:59.598486 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7469acd5-3c62-4625-9f13-8d468d6b9959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:01:59.654182 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1230/2430] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:01:59.662366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2b310-c3f5-4a6f-a57f-d31857ad813d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:59.662651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2b310-c3f5-4a6f-a57f-d31857ad813d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:59.662920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2b310-c3f5-4a6f-a57f-d31857ad813d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:59.683210 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f8d2b310-c3f5-4a6f-a57f-d31857ad813d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:01:59.684439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f8d2b310-c3f5-4a6f-a57f-d31857ad813d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:01:59.692218 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1201/2431] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:01:59.696504 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469acd5-3c62-4625-9f13-8d468d6b9959 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:01:59.706214 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469acd5-3c62-4625-9f13-8d468d6b9959 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:01:59.710682 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38693d-2343-4aa8-8589-f91f4cbe9f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:01:59.711299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38693d-2343-4aa8-8589-f91f4cbe9f95 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:59.711575 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38693d-2343-4aa8-8589-f91f4cbe9f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:59.711842 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38693d-2343-4aa8-8589-f91f4cbe9f95 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:59.712484 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38693d-2343-4aa8-8589-f91f4cbe9f95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:01:59.712781 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38693d-2343-4aa8-8589-f91f4cbe9f95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:01:59.714384 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38693d-2343-4aa8-8589-f91f4cbe9f95 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:01:59.726288 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1231/2432] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:01:59.730626 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2b310-c3f5-4a6f-a57f-d31857ad813d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:01:59.740623 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2b310-c3f5-4a6f-a57f-d31857ad813d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:01:59.745757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a3693-f7fe-4ee5-95af-6be43a136510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:01:59.746435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a3693-f7fe-4ee5-95af-6be43a136510 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:59.746817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a3693-f7fe-4ee5-95af-6be43a136510 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:59.747229 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a3693-f7fe-4ee5-95af-6be43a136510 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:59.748031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a3693-f7fe-4ee5-95af-6be43a136510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:01:59.750189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a3693-f7fe-4ee5-95af-6be43a136510 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:01:59.765574 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:01:59.863202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1202/2433] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:01:59 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:01:59.865407 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38693d-2343-4aa8-8589-f91f4cbe9f95 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:01:59.876210 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38693d-2343-4aa8-8589-f91f4cbe9f95 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:01:59.936950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ad057-0f6d-4b1b-9e7c-c892df122c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:01:59.937545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ad057-0f6d-4b1b-9e7c-c892df122c34 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:01:59.937824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ad057-0f6d-4b1b-9e7c-c892df122c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:01:59.938095 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ad057-0f6d-4b1b-9e7c-c892df122c34 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:01:59.938715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ad057-0f6d-4b1b-9e7c-c892df122c34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:01:59.940705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ad057-0f6d-4b1b-9e7c-c892df122c34 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:01:59.949116 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:00.046554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1232/2434] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:01:59 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:00.050748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a3693-f7fe-4ee5-95af-6be43a136510 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:00.061419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a3693-f7fe-4ee5-95af-6be43a136510 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:00.066286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40f310-d1fc-417c-a056-37ec7612ec27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:00.067540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40f310-d1fc-417c-a056-37ec7612ec27 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:00.067813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40f310-d1fc-417c-a056-37ec7612ec27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:00.068078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40f310-d1fc-417c-a056-37ec7612ec27 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:00.068867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40f310-d1fc-417c-a056-37ec7612ec27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:00.070463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40f310-d1fc-417c-a056-37ec7612ec27 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:00.081145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1203/2435] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:00.085124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ad057-0f6d-4b1b-9e7c-c892df122c34 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:00.095418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ad057-0f6d-4b1b-9e7c-c892df122c34 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:00.100552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4b71a-2f4a-4b8b-8fdc-4527ca006234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:00.101195 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4b71a-2f4a-4b8b-8fdc-4527ca006234 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:00.101472 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4b71a-2f4a-4b8b-8fdc-4527ca006234 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:00.101743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4b71a-2f4a-4b8b-8fdc-4527ca006234 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:00.102386 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4b71a-2f4a-4b8b-8fdc-4527ca006234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:00.104037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4b71a-2f4a-4b8b-8fdc-4527ca006234 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:00.115180 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1233/2436] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:00.119271 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40f310-d1fc-417c-a056-37ec7612ec27 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:00.130261 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40f310-d1fc-417c-a056-37ec7612ec27 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:00.135005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6fe08e-0530-4052-8c6a-fc8826bdd3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:00.139564 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fe08e-0530-4052-8c6a-fc8826bdd3da admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:00.139564 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fe08e-0530-4052-8c6a-fc8826bdd3da admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:00.139564 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fe08e-0530-4052-8c6a-fc8826bdd3da admin admin] PATH_INFO: `/v3/projects/42d43e5630bf423aad1ae87cc563b5e5/users/ae244a27b25b433ca86cecf2e7dddfe9/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:00.150588 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6fe08e-0530-4052-8c6a-fc8826bdd3da admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d43e5630bf423aad1ae87cc563b5e5, user_id=ae244a27b25b433ca86cecf2e7dddfe9, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:00.153272 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6fe08e-0530-4052-8c6a-fc8826bdd3da admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:00.220869 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1204/2437] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:00 2022] PUT /identity/v3/projects/42d43e5630bf423aad1ae87cc563b5e5/users/ae244a27b25b433ca86cecf2e7dddfe9/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:00.232567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cff4a-2c6d-4ce9-8f9a-965925dc5c50 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:00.232855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cff4a-2c6d-4ce9-8f9a-965925dc5c50 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:00.233131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cff4a-2c6d-4ce9-8f9a-965925dc5c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:00.261039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-5e4cff4a-2c6d-4ce9-8f9a-965925dc5c50 None None] MFA Rules not processed for user `ae244a27b25b433ca86cecf2e7dddfe9`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:00.262241 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-5e4cff4a-2c6d-4ce9-8f9a-965925dc5c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:00.324655 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1234/2438] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:00.335844 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6fe08e-0530-4052-8c6a-fc8826bdd3da admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:00.337473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-be6fe08e-0530-4052-8c6a-fc8826bdd3da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:00.409702 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6fe08e-0530-4052-8c6a-fc8826bdd3da admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:00.415784 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285a430-40ee-4353-b985-b0704fbe7cc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:00.416405 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285a430-40ee-4353-b985-b0704fbe7cc4 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:00.416664 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285a430-40ee-4353-b985-b0704fbe7cc4 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:00.416936 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285a430-40ee-4353-b985-b0704fbe7cc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:00.424569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285a430-40ee-4353-b985-b0704fbe7cc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:00.426730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285a430-40ee-4353-b985-b0704fbe7cc4 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:00.435536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1205/2439] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:02.654501 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cff4a-2c6d-4ce9-8f9a-965925dc5c50 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:02.667219 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cff4a-2c6d-4ce9-8f9a-965925dc5c50 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:02.672860 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b608c-caa0-4b70-af86-ec00d023d31e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:02.673482 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b608c-caa0-4b70-af86-ec00d023d31e service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:02.673759 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b608c-caa0-4b70-af86-ec00d023d31e service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:02.674032 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b608c-caa0-4b70-af86-ec00d023d31e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:02.675866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-894b608c-caa0-4b70-af86-ec00d023d31e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:02.736525 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b608c-caa0-4b70-af86-ec00d023d31e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:02.738114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b608c-caa0-4b70-af86-ec00d023d31e service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:02.747953 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1235/2440] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:03.641371 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285a430-40ee-4353-b985-b0704fbe7cc4 service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:03.710138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285a430-40ee-4353-b985-b0704fbe7cc4 service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:03.715063 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12e248-9214-48d1-a759-4cff17f694bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:03.715819 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12e248-9214-48d1-a759-4cff17f694bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:03.716097 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12e248-9214-48d1-a759-4cff17f694bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:03.716392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12e248-9214-48d1-a759-4cff17f694bc admin admin] PATH_INFO: `/v3/users/ae244a27b25b433ca86cecf2e7dddfe9` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:03.717585 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12e248-9214-48d1-a759-4cff17f694bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae244a27b25b433ca86cecf2e7dddfe9)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:03.719606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12e248-9214-48d1-a759-4cff17f694bc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:03.932628 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ed12e248-9214-48d1-a759-4cff17f694bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae244a27b25b433ca86cecf2e7dddfe9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:03.933986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:03.940508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ed12e248-9214-48d1-a759-4cff17f694bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae244a27b25b433ca86cecf2e7dddfe9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:03.941190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ed12e248-9214-48d1-a759-4cff17f694bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae244a27b25b433ca86cecf2e7dddfe9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:04.029319 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ed12e248-9214-48d1-a759-4cff17f694bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae244a27b25b433ca86cecf2e7dddfe9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:04.042292 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ed12e248-9214-48d1-a759-4cff17f694bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae244a27b25b433ca86cecf2e7dddfe9'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:04.048066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1206/2441] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:02:03 2022] DELETE /identity/v3/users/ae244a27b25b433ca86cecf2e7dddfe9 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:04.052620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63138ec0-154e-4313-a713-38d4ad28b839 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:04.053102 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63138ec0-154e-4313-a713-38d4ad28b839 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:04.053198 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63138ec0-154e-4313-a713-38d4ad28b839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:04.072680 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-63138ec0-154e-4313-a713-38d4ad28b839 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:04.073958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-63138ec0-154e-4313-a713-38d4ad28b839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:04.140782 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1236/2442] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:04.150461 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12e248-9214-48d1-a759-4cff17f694bc admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:04.151964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ed12e248-9214-48d1-a759-4cff17f694bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:04.208629 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12e248-9214-48d1-a759-4cff17f694bc admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:04.212821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950771f-2b34-4636-8e90-2c0a3f151c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:04.213414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950771f-2b34-4636-8e90-2c0a3f151c27 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:04.213695 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950771f-2b34-4636-8e90-2c0a3f151c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:04.213965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950771f-2b34-4636-8e90-2c0a3f151c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:04.222974 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950771f-2b34-4636-8e90-2c0a3f151c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:04.224435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950771f-2b34-4636-8e90-2c0a3f151c27 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:04.234503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1207/2443] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:04.685272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63138ec0-154e-4313-a713-38d4ad28b839 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:04.686707 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-63138ec0-154e-4313-a713-38d4ad28b839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:04.699067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63138ec0-154e-4313-a713-38d4ad28b839 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:04.703739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b749037-63df-4077-913d-61772e6e13b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:04.704382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b749037-63df-4077-913d-61772e6e13b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:04.704660 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b749037-63df-4077-913d-61772e6e13b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:04.704928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b749037-63df-4077-913d-61772e6e13b6 admin admin] PATH_INFO: `/v3/projects/42d43e5630bf423aad1ae87cc563b5e5` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:04.706045 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b749037-63df-4077-913d-61772e6e13b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42d43e5630bf423aad1ae87cc563b5e5)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:04.707985 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b749037-63df-4077-913d-61772e6e13b6 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:04.713273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:04.807477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-4b749037-63df-4077-913d-61772e6e13b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42d43e5630bf423aad1ae87cc563b5e5', 'request_id': 'req-4b749037-63df-4077-913d-61772e6e13b6'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:04.813110 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-4b749037-63df-4077-913d-61772e6e13b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42d43e5630bf423aad1ae87cc563b5e5', 'request_id': 'req-4b749037-63df-4077-913d-61772e6e13b6'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:04.900589 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-4b749037-63df-4077-913d-61772e6e13b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42d43e5630bf423aad1ae87cc563b5e5', 'request_id': 'req-4b749037-63df-4077-913d-61772e6e13b6'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:04.912888 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-4b749037-63df-4077-913d-61772e6e13b6 admin admin] The token cache is being invalidate because project 42d43e5630bf423aad1ae87cc563b5e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:02:04.913199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-4b749037-63df-4077-913d-61772e6e13b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:04.913925 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-4b749037-63df-4077-913d-61772e6e13b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:04.915346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1237/2444] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:02:04 2022] DELETE /identity/v3/projects/42d43e5630bf423aad1ae87cc563b5e5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:04.920223 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b10ec-89a3-4d3f-b586-221382683d45 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:04.920645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b10ec-89a3-4d3f-b586-221382683d45 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:04.921011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b10ec-89a3-4d3f-b586-221382683d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:04.941711 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-586b10ec-89a3-4d3f-b586-221382683d45 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:04.942902 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-586b10ec-89a3-4d3f-b586-221382683d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:05.002806 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1208/2445] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:05.011426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba0693-f175-431e-9b9f-15b8023a4b92 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:05.011709 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba0693-f175-431e-9b9f-15b8023a4b92 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:05.011982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba0693-f175-431e-9b9f-15b8023a4b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:05.031337 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-f0ba0693-f175-431e-9b9f-15b8023a4b92 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:05.034346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-f0ba0693-f175-431e-9b9f-15b8023a4b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:05.040721 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1238/2446] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:02:05.044928 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b10ec-89a3-4d3f-b586-221382683d45 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:05.055182 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b10ec-89a3-4d3f-b586-221382683d45 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:05.062842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e6d27-0e73-4bfa-8aec-078b45be52bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:05.062842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e6d27-0e73-4bfa-8aec-078b45be52bc admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:05.062842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e6d27-0e73-4bfa-8aec-078b45be52bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:05.062842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e6d27-0e73-4bfa-8aec-078b45be52bc admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:05.063270 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e6d27-0e73-4bfa-8aec-078b45be52bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:02:05.063270 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e6d27-0e73-4bfa-8aec-078b45be52bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:05.063648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e6d27-0e73-4bfa-8aec-078b45be52bc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:05.075763 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1209/2447] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:02:05.080139 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba0693-f175-431e-9b9f-15b8023a4b92 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:05.089450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba0693-f175-431e-9b9f-15b8023a4b92 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:05.094170 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df592593-9eeb-4389-89c2-747d1ab5f19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:05.094759 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df592593-9eeb-4389-89c2-747d1ab5f19c admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:05.095035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df592593-9eeb-4389-89c2-747d1ab5f19c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:05.095334 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df592593-9eeb-4389-89c2-747d1ab5f19c admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:05.095959 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df592593-9eeb-4389-89c2-747d1ab5f19c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:05.097841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df592593-9eeb-4389-89c2-747d1ab5f19c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:05.185818 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1239/2448] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:02:05 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:02:05.195489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e6d27-0e73-4bfa-8aec-078b45be52bc admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:05.205708 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e6d27-0e73-4bfa-8aec-078b45be52bc admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:05.256888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:05.267726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832b7cf-43e7-414c-838b-9f4d877dd51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:05.271426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832b7cf-43e7-414c-838b-9f4d877dd51d admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:05.271426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832b7cf-43e7-414c-838b-9f4d877dd51d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:05.271426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832b7cf-43e7-414c-838b-9f4d877dd51d admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:05.271426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6832b7cf-43e7-414c-838b-9f4d877dd51d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:05.272296 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6832b7cf-43e7-414c-838b-9f4d877dd51d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:05.389705 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1210/2449] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:02:05 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:05.392577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df592593-9eeb-4389-89c2-747d1ab5f19c admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:05.404677 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df592593-9eeb-4389-89c2-747d1ab5f19c admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:05.409397 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6f84f-a840-4dd7-b5e4-6fb4fc8849ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:05.409999 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6f84f-a840-4dd7-b5e4-6fb4fc8849ad admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:05.410280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6f84f-a840-4dd7-b5e4-6fb4fc8849ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:05.410555 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6f84f-a840-4dd7-b5e4-6fb4fc8849ad admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:05.411254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6f84f-a840-4dd7-b5e4-6fb4fc8849ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:05.413057 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6f84f-a840-4dd7-b5e4-6fb4fc8849ad admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:05.430358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1240/2450] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:02:05.431412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832b7cf-43e7-414c-838b-9f4d877dd51d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:05.442094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832b7cf-43e7-414c-838b-9f4d877dd51d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:05.448162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa95517-b816-463a-8e0c-0b6fc2ad2881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:05.448769 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa95517-b816-463a-8e0c-0b6fc2ad2881 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:05.449190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa95517-b816-463a-8e0c-0b6fc2ad2881 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:05.449463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa95517-b816-463a-8e0c-0b6fc2ad2881 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:05.450190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa95517-b816-463a-8e0c-0b6fc2ad2881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:05.451936 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa95517-b816-463a-8e0c-0b6fc2ad2881 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:05.464004 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1211/2451] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:02:05.468443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6f84f-a840-4dd7-b5e4-6fb4fc8849ad admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:05.479244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6f84f-a840-4dd7-b5e4-6fb4fc8849ad admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:05.484075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25506673-eb3f-4f0c-9224-dc1a9bb4523d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:05.484735 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25506673-eb3f-4f0c-9224-dc1a9bb4523d admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:05.485014 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25506673-eb3f-4f0c-9224-dc1a9bb4523d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:05.485286 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25506673-eb3f-4f0c-9224-dc1a9bb4523d admin admin] PATH_INFO: `/v3/projects/e54ffa294f874681964ce5ac6ef70fef/users/853068bd37b84fb2bbd37598b3c7bf04/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:05.504652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25506673-eb3f-4f0c-9224-dc1a9bb4523d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e54ffa294f874681964ce5ac6ef70fef, user_id=853068bd37b84fb2bbd37598b3c7bf04, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:05.508648 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25506673-eb3f-4f0c-9224-dc1a9bb4523d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:05.588995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1241/2452] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:05 2022] PUT /identity/v3/projects/e54ffa294f874681964ce5ac6ef70fef/users/853068bd37b84fb2bbd37598b3c7bf04/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:05.599775 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beb108-eca0-4c9d-b30f-8b6bba07f4f8 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:05.600130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beb108-eca0-4c9d-b30f-8b6bba07f4f8 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:05.600406 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beb108-eca0-4c9d-b30f-8b6bba07f4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:05.628451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-93beb108-eca0-4c9d-b30f-8b6bba07f4f8 None None] MFA Rules not processed for user `853068bd37b84fb2bbd37598b3c7bf04`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:05.629575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-93beb108-eca0-4c9d-b30f-8b6bba07f4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:05.694159 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1212/2453] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:05.699420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25506673-eb3f-4f0c-9224-dc1a9bb4523d admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:05.761496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25506673-eb3f-4f0c-9224-dc1a9bb4523d admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:05.766171 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574a6e0-539e-4e08-b8be-e2d25961375e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:05.766759 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574a6e0-539e-4e08-b8be-e2d25961375e admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:05.767037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574a6e0-539e-4e08-b8be-e2d25961375e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:05.773213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574a6e0-539e-4e08-b8be-e2d25961375e admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:05.773213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574a6e0-539e-4e08-b8be-e2d25961375e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:05.773213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574a6e0-539e-4e08-b8be-e2d25961375e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:05.850981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1242/2454] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:02:05 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:05.855030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93beb108-eca0-4c9d-b30f-8b6bba07f4f8 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:05.865781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93beb108-eca0-4c9d-b30f-8b6bba07f4f8 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:05.925315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44856ed0-6351-4991-8e57-67bf139e7580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:05.926134 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44856ed0-6351-4991-8e57-67bf139e7580 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:05.926346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44856ed0-6351-4991-8e57-67bf139e7580 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:05.926647 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44856ed0-6351-4991-8e57-67bf139e7580 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:05.927378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44856ed0-6351-4991-8e57-67bf139e7580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:05.930125 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44856ed0-6351-4991-8e57-67bf139e7580 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:06.078443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1213/2455] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:02:05 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:06.082599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574a6e0-539e-4e08-b8be-e2d25961375e admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:06.092841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574a6e0-539e-4e08-b8be-e2d25961375e admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:06.097372 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4a94b-56d4-4e8d-a16a-0c3ef5b048ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:06.097954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4a94b-56d4-4e8d-a16a-0c3ef5b048ce admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:06.098233 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4a94b-56d4-4e8d-a16a-0c3ef5b048ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:06.098500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4a94b-56d4-4e8d-a16a-0c3ef5b048ce admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:06.099187 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4a94b-56d4-4e8d-a16a-0c3ef5b048ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:06.101503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4a94b-56d4-4e8d-a16a-0c3ef5b048ce admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:06.106675 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:06.112135 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1243/2456] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:06.117051 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44856ed0-6351-4991-8e57-67bf139e7580 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:06.127183 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44856ed0-6351-4991-8e57-67bf139e7580 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:06.131844 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488453b-cba2-48b2-be23-220f9d96ca78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:06.132484 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488453b-cba2-48b2-be23-220f9d96ca78 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:06.132765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488453b-cba2-48b2-be23-220f9d96ca78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:06.133036 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488453b-cba2-48b2-be23-220f9d96ca78 admin admin] PATH_INFO: `/v3/projects/b15c0d91a8104e488f924383d231e3cf/users/dccfbc3f1b434c8587b8aea4104091c7/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:06.147722 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488453b-cba2-48b2-be23-220f9d96ca78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b15c0d91a8104e488f924383d231e3cf, user_id=dccfbc3f1b434c8587b8aea4104091c7, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:06.150341 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488453b-cba2-48b2-be23-220f9d96ca78 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:06.226364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1214/2457] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:06 2022] PUT /identity/v3/projects/b15c0d91a8104e488f924383d231e3cf/users/dccfbc3f1b434c8587b8aea4104091c7/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:06.230231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4a94b-56d4-4e8d-a16a-0c3ef5b048ce admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:06.240620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4a94b-56d4-4e8d-a16a-0c3ef5b048ce admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:06.296620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f5656-5474-4bea-9723-ef5944355680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:06.297329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f5656-5474-4bea-9723-ef5944355680 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:06.297983 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f5656-5474-4bea-9723-ef5944355680 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:06.298319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f5656-5474-4bea-9723-ef5944355680 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:06.299215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f5656-5474-4bea-9723-ef5944355680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:06.301027 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f5656-5474-4bea-9723-ef5944355680 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:06.312008 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1244/2458] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:06.316179 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488453b-cba2-48b2-be23-220f9d96ca78 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:06.326347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488453b-cba2-48b2-be23-220f9d96ca78 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:06.330979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15864fd3-5501-4124-accf-fa63ae90756f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:06.331651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15864fd3-5501-4124-accf-fa63ae90756f admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:06.331931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15864fd3-5501-4124-accf-fa63ae90756f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:06.332205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15864fd3-5501-4124-accf-fa63ae90756f admin admin] PATH_INFO: `/v3/projects/b15c0d91a8104e488f924383d231e3cf/users/dccfbc3f1b434c8587b8aea4104091c7/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:06.334111 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15864fd3-5501-4124-accf-fa63ae90756f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b15c0d91a8104e488f924383d231e3cf, user_id=dccfbc3f1b434c8587b8aea4104091c7, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:06.336818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15864fd3-5501-4124-accf-fa63ae90756f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:06.346819 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1215/2459] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:06 2022] PUT /identity/v3/projects/b15c0d91a8104e488f924383d231e3cf/users/dccfbc3f1b434c8587b8aea4104091c7/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:06.358880 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5347ad-fec4-4464-88fa-c32ee31e1b84 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:06.359189 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5347ad-fec4-4464-88fa-c32ee31e1b84 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:06.359465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5347ad-fec4-4464-88fa-c32ee31e1b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:06.387317 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-4f5347ad-fec4-4464-88fa-c32ee31e1b84 None None] MFA Rules not processed for user `dccfbc3f1b434c8587b8aea4104091c7`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:06.388526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-4f5347ad-fec4-4464-88fa-c32ee31e1b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:06.458907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1245/2460] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:06.470884 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15864fd3-5501-4124-accf-fa63ae90756f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:06.479648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-15864fd3-5501-4124-accf-fa63ae90756f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:06.493013 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:06.530947 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15864fd3-5501-4124-accf-fa63ae90756f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:06.534568 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffd805-834f-45d4-9976-ed163420c6c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:06.535192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffd805-834f-45d4-9976-ed163420c6c5 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:06.535468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffd805-834f-45d4-9976-ed163420c6c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:06.535734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffd805-834f-45d4-9976-ed163420c6c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:06.544396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffd805-834f-45d4-9976-ed163420c6c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:06.545840 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffd805-834f-45d4-9976-ed163420c6c5 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:06.555487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1216/2461] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:06 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:06.843111 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5347ad-fec4-4464-88fa-c32ee31e1b84 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:06.853436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5347ad-fec4-4464-88fa-c32ee31e1b84 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:06.857159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bc713-a018-40f8-a11b-11f55885f148 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:06.857756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bc713-a018-40f8-a11b-11f55885f148 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:06.858033 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bc713-a018-40f8-a11b-11f55885f148 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:06.858304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bc713-a018-40f8-a11b-11f55885f148 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:06.914870 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bc713-a018-40f8-a11b-11f55885f148 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:06.916366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bc713-a018-40f8-a11b-11f55885f148 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:06.925229 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1246/2462] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:12.420565 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffd805-834f-45d4-9976-ed163420c6c5 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:12.422054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-84ffd805-834f-45d4-9976-ed163420c6c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:12.488119 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffd805-834f-45d4-9976-ed163420c6c5 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:12.492588 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea888fea-2630-46e2-964d-b0b1246e619c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:12.493205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea888fea-2630-46e2-964d-b0b1246e619c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:12.493480 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea888fea-2630-46e2-964d-b0b1246e619c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:12.493749 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea888fea-2630-46e2-964d-b0b1246e619c admin admin] PATH_INFO: `/v3/users/d7dd8c6c2ba8426383bbb86b2969d43a` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:12.494857 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea888fea-2630-46e2-964d-b0b1246e619c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7dd8c6c2ba8426383bbb86b2969d43a)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:12.496799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea888fea-2630-46e2-964d-b0b1246e619c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:12.686315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ea888fea-2630-46e2-964d-b0b1246e619c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7dd8c6c2ba8426383bbb86b2969d43a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:12.693214 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ea888fea-2630-46e2-964d-b0b1246e619c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7dd8c6c2ba8426383bbb86b2969d43a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:12.693956 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ea888fea-2630-46e2-964d-b0b1246e619c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7dd8c6c2ba8426383bbb86b2969d43a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:12.757394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ea888fea-2630-46e2-964d-b0b1246e619c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7dd8c6c2ba8426383bbb86b2969d43a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:12.769400 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ea888fea-2630-46e2-964d-b0b1246e619c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7dd8c6c2ba8426383bbb86b2969d43a'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:12.775669 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1217/2463] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:02:12 2022] DELETE /identity/v3/users/d7dd8c6c2ba8426383bbb86b2969d43a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:13.054373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bc713-a018-40f8-a11b-11f55885f148 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:13.065546 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bc713-a018-40f8-a11b-11f55885f148 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:13.085343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:13.125934 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc75129-7c28-499b-b4b9-533758149ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:13.126557 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc75129-7c28-499b-b4b9-533758149ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:13.126842 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc75129-7c28-499b-b4b9-533758149ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:13.127123 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc75129-7c28-499b-b4b9-533758149ca9 admin admin] PATH_INFO: `/v3/projects/557a5dfba4324952bcdf72f58da8a0dc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:13.128271 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc75129-7c28-499b-b4b9-533758149ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=557a5dfba4324952bcdf72f58da8a0dc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:13.130179 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc75129-7c28-499b-b4b9-533758149ca9 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:13.315128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0fc75129-7c28-499b-b4b9-533758149ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '557a5dfba4324952bcdf72f58da8a0dc', 'request_id': 'req-0fc75129-7c28-499b-b4b9-533758149ca9'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:13.321336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0fc75129-7c28-499b-b4b9-533758149ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '557a5dfba4324952bcdf72f58da8a0dc', 'request_id': 'req-0fc75129-7c28-499b-b4b9-533758149ca9'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:13.397683 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0fc75129-7c28-499b-b4b9-533758149ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '557a5dfba4324952bcdf72f58da8a0dc', 'request_id': 'req-0fc75129-7c28-499b-b4b9-533758149ca9'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:13.408858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0fc75129-7c28-499b-b4b9-533758149ca9 admin admin] The token cache is being invalidate because project 557a5dfba4324952bcdf72f58da8a0dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:02:13.409149 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0fc75129-7c28-499b-b4b9-533758149ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:13.409778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0fc75129-7c28-499b-b4b9-533758149ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:13.411281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1247/2464] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:02:13 2022] DELETE /identity/v3/projects/557a5dfba4324952bcdf72f58da8a0dc => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:13.416137 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedad572-9011-4cbb-a0f0-2817632e5a23 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:13.416426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedad572-9011-4cbb-a0f0-2817632e5a23 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:13.416702 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedad572-9011-4cbb-a0f0-2817632e5a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:13.436331 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bedad572-9011-4cbb-a0f0-2817632e5a23 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:13.437520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bedad572-9011-4cbb-a0f0-2817632e5a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:13.469937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:13.496250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1218/2465] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:13.562596 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91fd4e-756a-455f-a47a-f98f32ded02b None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:13.562877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91fd4e-756a-455f-a47a-f98f32ded02b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:13.563186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91fd4e-756a-455f-a47a-f98f32ded02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:13.583239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-cb91fd4e-756a-455f-a47a-f98f32ded02b None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:13.584379 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-cb91fd4e-756a-455f-a47a-f98f32ded02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:13.592620 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1248/2466] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:13.597045 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedad572-9011-4cbb-a0f0-2817632e5a23 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:13.607408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedad572-9011-4cbb-a0f0-2817632e5a23 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:13.612238 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43932c15-62a8-49df-8d24-788641565f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:13.612830 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43932c15-62a8-49df-8d24-788641565f39 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:13.613103 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43932c15-62a8-49df-8d24-788641565f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:13.613369 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43932c15-62a8-49df-8d24-788641565f39 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:13.614011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43932c15-62a8-49df-8d24-788641565f39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:02:13.614308 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43932c15-62a8-49df-8d24-788641565f39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:13.615941 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43932c15-62a8-49df-8d24-788641565f39 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:13.628693 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1219/2467] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:02:13.632875 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91fd4e-756a-455f-a47a-f98f32ded02b None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:13.642206 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91fd4e-756a-455f-a47a-f98f32ded02b None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:13.647039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeeab28-84c4-4d92-a0a3-a453b39f43a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:13.647667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeeab28-84c4-4d92-a0a3-a453b39f43a8 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:13.647953 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeeab28-84c4-4d92-a0a3-a453b39f43a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:13.648265 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeeab28-84c4-4d92-a0a3-a453b39f43a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:13.648897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeeab28-84c4-4d92-a0a3-a453b39f43a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:13.650787 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeeab28-84c4-4d92-a0a3-a453b39f43a8 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:13.776863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1249/2468] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:02:13 2022] POST /identity/v3/projects => generated 397 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:13.780949 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43932c15-62a8-49df-8d24-788641565f39 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:13.792085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43932c15-62a8-49df-8d24-788641565f39 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:13.849085 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98cbf8-c9a2-4f30-9ae9-8d6d606cff42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:13.849954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98cbf8-c9a2-4f30-9ae9-8d6d606cff42 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:13.850234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98cbf8-c9a2-4f30-9ae9-8d6d606cff42 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:13.850511 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98cbf8-c9a2-4f30-9ae9-8d6d606cff42 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:13.851679 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b98cbf8-c9a2-4f30-9ae9-8d6d606cff42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:13.853588 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b98cbf8-c9a2-4f30-9ae9-8d6d606cff42 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:13.945920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1220/2469] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:02:13 2022] POST /identity/v3/users => generated 299 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:13.950025 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeeab28-84c4-4d92-a0a3-a453b39f43a8 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:13.960277 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeeab28-84c4-4d92-a0a3-a453b39f43a8 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:13.965346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49406271-d000-448f-b7d9-b92adc14e0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:13.965926 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49406271-d000-448f-b7d9-b92adc14e0da admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:13.966200 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49406271-d000-448f-b7d9-b92adc14e0da admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:13.966468 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49406271-d000-448f-b7d9-b92adc14e0da admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:13.967123 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49406271-d000-448f-b7d9-b92adc14e0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:13.968701 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49406271-d000-448f-b7d9-b92adc14e0da admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:13.979428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1250/2470] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:02:13.983039 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98cbf8-c9a2-4f30-9ae9-8d6d606cff42 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:13.993024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98cbf8-c9a2-4f30-9ae9-8d6d606cff42 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:13.997744 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ead9a8-609e-4583-a105-1f31b10cfa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:13.998367 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ead9a8-609e-4583-a105-1f31b10cfa46 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:13.998648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ead9a8-609e-4583-a105-1f31b10cfa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:13.998920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ead9a8-609e-4583-a105-1f31b10cfa46 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:13.999594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ead9a8-609e-4583-a105-1f31b10cfa46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:14.001187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ead9a8-609e-4583-a105-1f31b10cfa46 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:14.012414 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1221/2471] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:14.016600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49406271-d000-448f-b7d9-b92adc14e0da admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:14.026939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49406271-d000-448f-b7d9-b92adc14e0da admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:14.031496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71717597-4474-4046-98f5-b5e8725664e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:14.032131 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71717597-4474-4046-98f5-b5e8725664e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:14.032404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71717597-4474-4046-98f5-b5e8725664e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:14.032670 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71717597-4474-4046-98f5-b5e8725664e2 admin admin] PATH_INFO: `/v3/projects/21e7b431fb2c49bd8b4b055c796c2816/users/dd74df25e5e34ec194dbd4ad37fb9bfd/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:14.047402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71717597-4474-4046-98f5-b5e8725664e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e7b431fb2c49bd8b4b055c796c2816, user_id=dd74df25e5e34ec194dbd4ad37fb9bfd, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:14.050048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71717597-4474-4046-98f5-b5e8725664e2 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:14.123781 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1251/2472] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:14 2022] PUT /identity/v3/projects/21e7b431fb2c49bd8b4b055c796c2816/users/dd74df25e5e34ec194dbd4ad37fb9bfd/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:14.133591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13683e87-c5bc-46d3-aa5b-2d1a02608f0a None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:14.134576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13683e87-c5bc-46d3-aa5b-2d1a02608f0a None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:14.134576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13683e87-c5bc-46d3-aa5b-2d1a02608f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:14.163059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-13683e87-c5bc-46d3-aa5b-2d1a02608f0a None None] MFA Rules not processed for user `dd74df25e5e34ec194dbd4ad37fb9bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:14.164963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-13683e87-c5bc-46d3-aa5b-2d1a02608f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:14.227846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1222/2473] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:14 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:14.239463 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71717597-4474-4046-98f5-b5e8725664e2 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:14.241402 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-71717597-4474-4046-98f5-b5e8725664e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:14.293717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:14.298814 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71717597-4474-4046-98f5-b5e8725664e2 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:14.302492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257bec3-0fb0-4e2a-b98e-e478b760f799 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:14.306473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257bec3-0fb0-4e2a-b98e-e478b760f799 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:14.306473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257bec3-0fb0-4e2a-b98e-e478b760f799 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:14.306473 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257bec3-0fb0-4e2a-b98e-e478b760f799 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:14.311735 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2257bec3-0fb0-4e2a-b98e-e478b760f799 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:14.313142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2257bec3-0fb0-4e2a-b98e-e478b760f799 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:14.322825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1252/2474] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:14 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:17.966033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13683e87-c5bc-46d3-aa5b-2d1a02608f0a None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:17.967443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-13683e87-c5bc-46d3-aa5b-2d1a02608f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:18.001990 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:18.031550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13683e87-c5bc-46d3-aa5b-2d1a02608f0a None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:18.036305 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad293ed4-4bf2-428a-af40-40500b884f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:18.036934 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad293ed4-4bf2-428a-af40-40500b884f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:18.037411 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad293ed4-4bf2-428a-af40-40500b884f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:18.037863 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad293ed4-4bf2-428a-af40-40500b884f5e admin admin] PATH_INFO: `/v3/users/853068bd37b84fb2bbd37598b3c7bf04` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:18.039470 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad293ed4-4bf2-428a-af40-40500b884f5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=853068bd37b84fb2bbd37598b3c7bf04)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:18.041379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad293ed4-4bf2-428a-af40-40500b884f5e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:18.222035 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ad293ed4-4bf2-428a-af40-40500b884f5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '853068bd37b84fb2bbd37598b3c7bf04'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:18.229521 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ad293ed4-4bf2-428a-af40-40500b884f5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '853068bd37b84fb2bbd37598b3c7bf04'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:18.230277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ad293ed4-4bf2-428a-af40-40500b884f5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '853068bd37b84fb2bbd37598b3c7bf04'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:18.340713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ad293ed4-4bf2-428a-af40-40500b884f5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '853068bd37b84fb2bbd37598b3c7bf04'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:18.353810 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ad293ed4-4bf2-428a-af40-40500b884f5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '853068bd37b84fb2bbd37598b3c7bf04'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:18.359640 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1223/2475] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:02:17 2022] DELETE /identity/v3/users/853068bd37b84fb2bbd37598b3c7bf04 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:18.364294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eb02b-b9a2-45fc-8fd2-074b3897da73 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:18.364727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eb02b-b9a2-45fc-8fd2-074b3897da73 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:18.364833 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eb02b-b9a2-45fc-8fd2-074b3897da73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:18.383904 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-001eb02b-b9a2-45fc-8fd2-074b3897da73 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:18.386359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-001eb02b-b9a2-45fc-8fd2-074b3897da73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:18.444587 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1253/2476] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:18.456495 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad293ed4-4bf2-428a-af40-40500b884f5e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:18.459312 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ad293ed4-4bf2-428a-af40-40500b884f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:18.517462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad293ed4-4bf2-428a-af40-40500b884f5e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:18.521280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93f39b-8a82-4893-9ad7-488030eb6fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:18.522130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93f39b-8a82-4893-9ad7-488030eb6fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:18.522418 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93f39b-8a82-4893-9ad7-488030eb6fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:18.522787 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93f39b-8a82-4893-9ad7-488030eb6fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:18.532810 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93f39b-8a82-4893-9ad7-488030eb6fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:18.534523 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93f39b-8a82-4893-9ad7-488030eb6fe9 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:18.545034 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1224/2477] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:18.962282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eb02b-b9a2-45fc-8fd2-074b3897da73 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:18.963794 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-001eb02b-b9a2-45fc-8fd2-074b3897da73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:18.976080 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eb02b-b9a2-45fc-8fd2-074b3897da73 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:18.981194 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:18.981826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:18.982102 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:18.982371 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b admin admin] PATH_INFO: `/v3/projects/e54ffa294f874681964ce5ac6ef70fef` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:18.983524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e54ffa294f874681964ce5ac6ef70fef)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:18.985423 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:19.115873 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e54ffa294f874681964ce5ac6ef70fef', 'request_id': 'req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:19.121228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e54ffa294f874681964ce5ac6ef70fef', 'request_id': 'req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:19.191822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e54ffa294f874681964ce5ac6ef70fef', 'request_id': 'req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:19.203653 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b admin admin] The token cache is being invalidate because project e54ffa294f874681964ce5ac6ef70fef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:02:19.208783 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:19.208783 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:19.208783 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1254/2478] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:02:18 2022] DELETE /identity/v3/projects/e54ffa294f874681964ce5ac6ef70fef => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:19.210010 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93f39b-8a82-4893-9ad7-488030eb6fe9 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:19.219799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93f39b-8a82-4893-9ad7-488030eb6fe9 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:19.270069 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:19.276681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7917b-ae06-44d2-a639-c1dc9531121a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:19.277302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7917b-ae06-44d2-a639-c1dc9531121a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:19.277580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7917b-ae06-44d2-a639-c1dc9531121a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:19.277849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7917b-ae06-44d2-a639-c1dc9531121a admin admin] PATH_INFO: `/v3/users/dccfbc3f1b434c8587b8aea4104091c7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:19.279012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7917b-ae06-44d2-a639-c1dc9531121a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dccfbc3f1b434c8587b8aea4104091c7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:19.283303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7917b-ae06-44d2-a639-c1dc9531121a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:19.499905 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-14d7917b-ae06-44d2-a639-c1dc9531121a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dccfbc3f1b434c8587b8aea4104091c7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:19.506692 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-14d7917b-ae06-44d2-a639-c1dc9531121a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dccfbc3f1b434c8587b8aea4104091c7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:19.508020 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-14d7917b-ae06-44d2-a639-c1dc9531121a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dccfbc3f1b434c8587b8aea4104091c7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:19.577717 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-14d7917b-ae06-44d2-a639-c1dc9531121a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dccfbc3f1b434c8587b8aea4104091c7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:19.593483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-14d7917b-ae06-44d2-a639-c1dc9531121a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dccfbc3f1b434c8587b8aea4104091c7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:19.599952 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1225/2479] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:02:19 2022] DELETE /identity/v3/users/dccfbc3f1b434c8587b8aea4104091c7 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:02:20.050863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:20.052350 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:20.065635 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad55c8-3c7b-4ae3-9bf8-d82220d4c15b admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:20.125484 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3547e27-4c1e-456f-b2d2-041532c7010c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:20.126106 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3547e27-4c1e-456f-b2d2-041532c7010c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:20.126381 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3547e27-4c1e-456f-b2d2-041532c7010c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:20.126651 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3547e27-4c1e-456f-b2d2-041532c7010c admin admin] PATH_INFO: `/v3/projects/b15c0d91a8104e488f924383d231e3cf` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:20.127824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3547e27-4c1e-456f-b2d2-041532c7010c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b15c0d91a8104e488f924383d231e3cf)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:20.129716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3547e27-4c1e-456f-b2d2-041532c7010c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:20.145322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:20.239562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a3547e27-4c1e-456f-b2d2-041532c7010c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b15c0d91a8104e488f924383d231e3cf', 'request_id': 'req-a3547e27-4c1e-456f-b2d2-041532c7010c'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:20.245045 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a3547e27-4c1e-456f-b2d2-041532c7010c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b15c0d91a8104e488f924383d231e3cf', 'request_id': 'req-a3547e27-4c1e-456f-b2d2-041532c7010c'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:20.314900 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a3547e27-4c1e-456f-b2d2-041532c7010c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b15c0d91a8104e488f924383d231e3cf', 'request_id': 'req-a3547e27-4c1e-456f-b2d2-041532c7010c'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:20.327140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a3547e27-4c1e-456f-b2d2-041532c7010c admin admin] The token cache is being invalidate because project b15c0d91a8104e488f924383d231e3cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:02:20.327958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a3547e27-4c1e-456f-b2d2-041532c7010c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:20.328828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a3547e27-4c1e-456f-b2d2-041532c7010c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:20.330290 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1255/2480] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:02:20 2022] DELETE /identity/v3/projects/b15c0d91a8104e488f924383d231e3cf => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:20.335088 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c790cd-c7e2-4925-90ce-aa713e78dc40 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:20.336239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c790cd-c7e2-4925-90ce-aa713e78dc40 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:20.336239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c790cd-c7e2-4925-90ce-aa713e78dc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:20.355709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-20c790cd-c7e2-4925-90ce-aa713e78dc40 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:20.356895 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-20c790cd-c7e2-4925-90ce-aa713e78dc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:20.414033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1226/2481] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:20.421691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477e6f8-1c13-425b-90ec-d27791101223 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:20.421978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477e6f8-1c13-425b-90ec-d27791101223 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:20.422253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477e6f8-1c13-425b-90ec-d27791101223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:20.442514 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-f477e6f8-1c13-425b-90ec-d27791101223 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:20.443738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-f477e6f8-1c13-425b-90ec-d27791101223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:20.452199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1256/2482] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:20.456291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c790cd-c7e2-4925-90ce-aa713e78dc40 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:20.466339 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c790cd-c7e2-4925-90ce-aa713e78dc40 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:20.470802 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61afe35-a8a1-4ee2-aa08-56c96134f9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:20.471424 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61afe35-a8a1-4ee2-aa08-56c96134f9ea admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:20.471702 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61afe35-a8a1-4ee2-aa08-56c96134f9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:20.471970 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61afe35-a8a1-4ee2-aa08-56c96134f9ea admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:20.472610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61afe35-a8a1-4ee2-aa08-56c96134f9ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:02:20.472909 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61afe35-a8a1-4ee2-aa08-56c96134f9ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:20.474749 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61afe35-a8a1-4ee2-aa08-56c96134f9ea admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:20.485972 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1227/2483] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:02:20.490299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477e6f8-1c13-425b-90ec-d27791101223 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:20.499838 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477e6f8-1c13-425b-90ec-d27791101223 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:20.504406 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a66e97-77bf-4e87-bf65-52acbce150d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:20.504988 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a66e97-77bf-4e87-bf65-52acbce150d5 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:20.505264 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a66e97-77bf-4e87-bf65-52acbce150d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:20.505535 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a66e97-77bf-4e87-bf65-52acbce150d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:20.506154 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a66e97-77bf-4e87-bf65-52acbce150d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:20.508584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a66e97-77bf-4e87-bf65-52acbce150d5 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:20.596561 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1257/2484] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:02:20 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:20.600957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61afe35-a8a1-4ee2-aa08-56c96134f9ea admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:20.610991 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61afe35-a8a1-4ee2-aa08-56c96134f9ea admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:20.666445 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c20b79-2f99-440d-9ab4-8393a4d3f11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:20.667032 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c20b79-2f99-440d-9ab4-8393a4d3f11d admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:20.667335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c20b79-2f99-440d-9ab4-8393a4d3f11d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:20.667601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c20b79-2f99-440d-9ab4-8393a4d3f11d admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:20.668232 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c20b79-2f99-440d-9ab4-8393a4d3f11d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:20.670152 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c20b79-2f99-440d-9ab4-8393a4d3f11d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:20.678905 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:20.762140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a66e97-77bf-4e87-bf65-52acbce150d5 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:20.763920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d4a66e97-77bf-4e87-bf65-52acbce150d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:20.782088 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1228/2485] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:02:20 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:20.786282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c20b79-2f99-440d-9ab4-8393a4d3f11d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:20.799573 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c20b79-2f99-440d-9ab4-8393a4d3f11d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:20.807065 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461db66-4b9b-4a83-ad37-469c6a79faa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:20.807582 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461db66-4b9b-4a83-ad37-469c6a79faa9 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:20.807976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461db66-4b9b-4a83-ad37-469c6a79faa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:20.808292 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461db66-4b9b-4a83-ad37-469c6a79faa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:20.808995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461db66-4b9b-4a83-ad37-469c6a79faa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:20.810811 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461db66-4b9b-4a83-ad37-469c6a79faa9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:20.823571 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1229/2486] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:02:20.828297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461db66-4b9b-4a83-ad37-469c6a79faa9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:20.838666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461db66-4b9b-4a83-ad37-469c6a79faa9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:20.844611 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f72d84-d9e0-4cd3-95ce-187835a73b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:20.845266 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f72d84-d9e0-4cd3-95ce-187835a73b67 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:20.845806 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f72d84-d9e0-4cd3-95ce-187835a73b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:20.846267 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f72d84-d9e0-4cd3-95ce-187835a73b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:20.847317 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f72d84-d9e0-4cd3-95ce-187835a73b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:20.848357 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a66e97-77bf-4e87-bf65-52acbce150d5 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:20.849021 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f72d84-d9e0-4cd3-95ce-187835a73b67 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:20.857042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a3ee5-abe5-4c87-8a4f-590c727d3405 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:20.857736 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3ee5-abe5-4c87-8a4f-590c727d3405 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:20.858016 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3ee5-abe5-4c87-8a4f-590c727d3405 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:20.858289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3ee5-abe5-4c87-8a4f-590c727d3405 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:20.860033 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-047a3ee5-abe5-4c87-8a4f-590c727d3405 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:20.863398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1230/2487] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:20.864689 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f72d84-d9e0-4cd3-95ce-187835a73b67 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:20.878141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f72d84-d9e0-4cd3-95ce-187835a73b67 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:20.885074 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6d9f1-03a5-4efa-85be-d4a49e56a511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:20.885969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6d9f1-03a5-4efa-85be-d4a49e56a511 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:20.886276 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6d9f1-03a5-4efa-85be-d4a49e56a511 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:20.886518 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6d9f1-03a5-4efa-85be-d4a49e56a511 admin admin] PATH_INFO: `/v3/projects/1181cb0087af4eaea5b135bca726725e/users/18f3356426a24e719b138332345799c1/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:20.902723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6d9f1-03a5-4efa-85be-d4a49e56a511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1181cb0087af4eaea5b135bca726725e, user_id=18f3356426a24e719b138332345799c1, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:20.905425 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6d9f1-03a5-4efa-85be-d4a49e56a511 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:20.924764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a3ee5-abe5-4c87-8a4f-590c727d3405 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:20.926452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a3ee5-abe5-4c87-8a4f-590c727d3405 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:20.936855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1258/2488] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:02:21.004657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1231/2489] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:20 2022] PUT /identity/v3/projects/1181cb0087af4eaea5b135bca726725e/users/18f3356426a24e719b138332345799c1/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:02:21.021408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c5d52-2654-40b2-a5e5-857754476377 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:21.021408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c5d52-2654-40b2-a5e5-857754476377 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:21.021408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c5d52-2654-40b2-a5e5-857754476377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:21.045644 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:21.059797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-485c5d52-2654-40b2-a5e5-857754476377 None None] MFA Rules not processed for user `18f3356426a24e719b138332345799c1`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:21.059797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-485c5d52-2654-40b2-a5e5-857754476377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:21.130039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1259/2490] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:21.134500 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6d9f1-03a5-4efa-85be-d4a49e56a511 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:21.212317 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6d9f1-03a5-4efa-85be-d4a49e56a511 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:21.216961 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab955b-8420-4b98-ba47-cfae5113c85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:21.217562 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab955b-8420-4b98-ba47-cfae5113c85b admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:21.221830 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab955b-8420-4b98-ba47-cfae5113c85b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:21.221830 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab955b-8420-4b98-ba47-cfae5113c85b admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:21.221830 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab955b-8420-4b98-ba47-cfae5113c85b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:21.221830 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab955b-8420-4b98-ba47-cfae5113c85b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:21.316661 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1232/2491] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:02:21 2022] POST /identity/v3/projects => generated 369 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:02:21.321205 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c5d52-2654-40b2-a5e5-857754476377 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:21.333463 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c5d52-2654-40b2-a5e5-857754476377 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:21.391356 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d85ee5-0875-4a0d-b326-8a70a5f3dbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:21.391881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d85ee5-0875-4a0d-b326-8a70a5f3dbd0 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:21.392158 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d85ee5-0875-4a0d-b326-8a70a5f3dbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:21.392424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d85ee5-0875-4a0d-b326-8a70a5f3dbd0 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:21.393091 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d85ee5-0875-4a0d-b326-8a70a5f3dbd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:21.395179 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d85ee5-0875-4a0d-b326-8a70a5f3dbd0 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:21.500995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1260/2492] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:02:21 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:21.502208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab955b-8420-4b98-ba47-cfae5113c85b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:21.513157 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab955b-8420-4b98-ba47-cfae5113c85b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:21.518426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc999e-d95e-49c1-b8cf-40d424591ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:21.519082 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc999e-d95e-49c1-b8cf-40d424591ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:21.519427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc999e-d95e-49c1-b8cf-40d424591ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:21.519699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc999e-d95e-49c1-b8cf-40d424591ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:21.520417 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc999e-d95e-49c1-b8cf-40d424591ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:21.522046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc999e-d95e-49c1-b8cf-40d424591ff1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:21.533595 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1233/2493] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:02:21.537978 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d85ee5-0875-4a0d-b326-8a70a5f3dbd0 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:21.548847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d85ee5-0875-4a0d-b326-8a70a5f3dbd0 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:21.553454 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ad22e-6ffa-420f-b093-21e760d41c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:21.554089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ad22e-6ffa-420f-b093-21e760d41c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:21.554361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ad22e-6ffa-420f-b093-21e760d41c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:21.554674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ad22e-6ffa-420f-b093-21e760d41c5e admin admin] PATH_INFO: `/v3/projects/4e0fe5e0b76048059ea58db6921661f1/users/575f0abf57694440b407907908bf9841/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:21.569128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ad22e-6ffa-420f-b093-21e760d41c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e0fe5e0b76048059ea58db6921661f1, user_id=575f0abf57694440b407907908bf9841, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:21.571934 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ad22e-6ffa-420f-b093-21e760d41c5e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:21.689404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1261/2494] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:21 2022] PUT /identity/v3/projects/4e0fe5e0b76048059ea58db6921661f1/users/575f0abf57694440b407907908bf9841/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:02:21.693387 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc999e-d95e-49c1-b8cf-40d424591ff1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:21.703587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc999e-d95e-49c1-b8cf-40d424591ff1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:21.766272 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719e5e9-b9a1-4c48-b06c-d5cac8ca525e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:21.766881 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719e5e9-b9a1-4c48-b06c-d5cac8ca525e admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:21.767199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719e5e9-b9a1-4c48-b06c-d5cac8ca525e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:21.767472 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719e5e9-b9a1-4c48-b06c-d5cac8ca525e admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:21.768115 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719e5e9-b9a1-4c48-b06c-d5cac8ca525e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:21.769713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719e5e9-b9a1-4c48-b06c-d5cac8ca525e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:21.780609 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1234/2495] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:21.784888 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ad22e-6ffa-420f-b093-21e760d41c5e admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:21.799650 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ad22e-6ffa-420f-b093-21e760d41c5e admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:21.800769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453e80a-76c1-4ef9-b690-b5c3f48d049f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:21.801440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453e80a-76c1-4ef9-b690-b5c3f48d049f admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:21.801714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453e80a-76c1-4ef9-b690-b5c3f48d049f admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:21.801986 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453e80a-76c1-4ef9-b690-b5c3f48d049f admin admin] PATH_INFO: `/v3/projects/4e0fe5e0b76048059ea58db6921661f1/users/575f0abf57694440b407907908bf9841/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:21.803921 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453e80a-76c1-4ef9-b690-b5c3f48d049f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e0fe5e0b76048059ea58db6921661f1, user_id=575f0abf57694440b407907908bf9841, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:21.806701 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453e80a-76c1-4ef9-b690-b5c3f48d049f admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:21.826431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1262/2496] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:21 2022] PUT /identity/v3/projects/4e0fe5e0b76048059ea58db6921661f1/users/575f0abf57694440b407907908bf9841/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:21.837347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97f1b3-5c59-4a87-a9a5-49985efbb181 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:21.837629 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97f1b3-5c59-4a87-a9a5-49985efbb181 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:21.837901 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97f1b3-5c59-4a87-a9a5-49985efbb181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:21.866274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bf97f1b3-5c59-4a87-a9a5-49985efbb181 None None] MFA Rules not processed for user `575f0abf57694440b407907908bf9841`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:21.868966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bf97f1b3-5c59-4a87-a9a5-49985efbb181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:21.871434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:21.933435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1235/2497] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:21.944420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453e80a-76c1-4ef9-b690-b5c3f48d049f admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:21.946606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-5453e80a-76c1-4ef9-b690-b5c3f48d049f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:22.004133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453e80a-76c1-4ef9-b690-b5c3f48d049f admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:22.008345 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdccbf-1ae2-4923-a477-a8b9fa86e9fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:22.008948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdccbf-1ae2-4923-a477-a8b9fa86e9fb service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:22.009222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdccbf-1ae2-4923-a477-a8b9fa86e9fb service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:22.009490 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdccbf-1ae2-4923-a477-a8b9fa86e9fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:22.016904 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:22.024080 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fdccbf-1ae2-4923-a477-a8b9fa86e9fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:22.024080 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fdccbf-1ae2-4923-a477-a8b9fa86e9fb service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:22.032057 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1263/2498] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:23.529239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97f1b3-5c59-4a87-a9a5-49985efbb181 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:23.530892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bf97f1b3-5c59-4a87-a9a5-49985efbb181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:23.609316 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97f1b3-5c59-4a87-a9a5-49985efbb181 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:23.615025 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995a6b4-14d4-44af-896c-e277147e1798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:23.617846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995a6b4-14d4-44af-896c-e277147e1798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:23.617846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995a6b4-14d4-44af-896c-e277147e1798 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:23.617846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995a6b4-14d4-44af-896c-e277147e1798 admin admin] PATH_INFO: `/v3/users/dd74df25e5e34ec194dbd4ad37fb9bfd` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:23.618295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995a6b4-14d4-44af-896c-e277147e1798 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd74df25e5e34ec194dbd4ad37fb9bfd)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:23.620573 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995a6b4-14d4-44af-896c-e277147e1798 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:23.939236 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2995a6b4-14d4-44af-896c-e277147e1798 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd74df25e5e34ec194dbd4ad37fb9bfd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:23.950059 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2995a6b4-14d4-44af-896c-e277147e1798 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd74df25e5e34ec194dbd4ad37fb9bfd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:23.950853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2995a6b4-14d4-44af-896c-e277147e1798 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd74df25e5e34ec194dbd4ad37fb9bfd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:24.029248 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2995a6b4-14d4-44af-896c-e277147e1798 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd74df25e5e34ec194dbd4ad37fb9bfd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:24.044770 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-2995a6b4-14d4-44af-896c-e277147e1798 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd74df25e5e34ec194dbd4ad37fb9bfd'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:24.050579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1236/2499] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:02:23 2022] DELETE /identity/v3/users/dd74df25e5e34ec194dbd4ad37fb9bfd => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:24.056319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a6246-5d94-4bcd-8aef-f0dfd22a4f6f None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:24.056319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a6246-5d94-4bcd-8aef-f0dfd22a4f6f None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:24.064418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a6246-5d94-4bcd-8aef-f0dfd22a4f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:24.076386 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-861a6246-5d94-4bcd-8aef-f0dfd22a4f6f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:24.077720 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-861a6246-5d94-4bcd-8aef-f0dfd22a4f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:24.129950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe1470-4554-4ad1-ab8e-9a9fd99f4b92 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:24.131671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe1470-4554-4ad1-ab8e-9a9fd99f4b92 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:24.131671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe1470-4554-4ad1-ab8e-9a9fd99f4b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:24.152093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1264/2500] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:24.154364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e5fe1470-4554-4ad1-ab8e-9a9fd99f4b92 None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:24.155998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e5fe1470-4554-4ad1-ab8e-9a9fd99f4b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:24.174671 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a6246-5d94-4bcd-8aef-f0dfd22a4f6f None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:24.175967 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:24.176260 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-861a6246-5d94-4bcd-8aef-f0dfd22a4f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:24.239726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1237/2501] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 30 15:02:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:02:24.257231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a6246-5d94-4bcd-8aef-f0dfd22a4f6f None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:24.274574 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b313f-4766-4097-8f16-07eccbc7804b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:24.275399 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b313f-4766-4097-8f16-07eccbc7804b service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:24.275882 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b313f-4766-4097-8f16-07eccbc7804b service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:24.276164 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b313f-4766-4097-8f16-07eccbc7804b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:24.290896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b313f-4766-4097-8f16-07eccbc7804b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:24.292431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b313f-4766-4097-8f16-07eccbc7804b service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:24.306120 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1265/2502] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:24.403276 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe1470-4554-4ad1-ab8e-9a9fd99f4b92 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:24.404943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e5fe1470-4554-4ad1-ab8e-9a9fd99f4b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:24.507260 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe1470-4554-4ad1-ab8e-9a9fd99f4b92 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:24.513512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da75d6b-3212-4d27-90ab-67b9dfa10fcb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:24.514262 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da75d6b-3212-4d27-90ab-67b9dfa10fcb service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:24.514610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da75d6b-3212-4d27-90ab-67b9dfa10fcb service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:24.514878 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da75d6b-3212-4d27-90ab-67b9dfa10fcb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:24.523957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da75d6b-3212-4d27-90ab-67b9dfa10fcb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:24.525547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da75d6b-3212-4d27-90ab-67b9dfa10fcb service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:24.534587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1238/2503] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 15:02:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:02:25.122380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b313f-4766-4097-8f16-07eccbc7804b service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:25.123897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-878b313f-4766-4097-8f16-07eccbc7804b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:25.148533 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b313f-4766-4097-8f16-07eccbc7804b service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:25.164577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45925b17-8d8a-41d1-990f-9c53c8ba0495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:25.165489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45925b17-8d8a-41d1-990f-9c53c8ba0495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:25.165946 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45925b17-8d8a-41d1-990f-9c53c8ba0495 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:25.166897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45925b17-8d8a-41d1-990f-9c53c8ba0495 admin admin] PATH_INFO: `/v3/projects/21e7b431fb2c49bd8b4b055c796c2816` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:25.168532 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45925b17-8d8a-41d1-990f-9c53c8ba0495 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21e7b431fb2c49bd8b4b055c796c2816)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:25.170727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45925b17-8d8a-41d1-990f-9c53c8ba0495 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:25.308813 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-45925b17-8d8a-41d1-990f-9c53c8ba0495 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21e7b431fb2c49bd8b4b055c796c2816', 'request_id': 'req-45925b17-8d8a-41d1-990f-9c53c8ba0495'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:25.318911 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-45925b17-8d8a-41d1-990f-9c53c8ba0495 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21e7b431fb2c49bd8b4b055c796c2816', 'request_id': 'req-45925b17-8d8a-41d1-990f-9c53c8ba0495'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:25.397287 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da75d6b-3212-4d27-90ab-67b9dfa10fcb service swift] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:25.398821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9da75d6b-3212-4d27-90ab-67b9dfa10fcb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:25.403426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-45925b17-8d8a-41d1-990f-9c53c8ba0495 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21e7b431fb2c49bd8b4b055c796c2816', 'request_id': 'req-45925b17-8d8a-41d1-990f-9c53c8ba0495'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:25.419474 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-45925b17-8d8a-41d1-990f-9c53c8ba0495 admin admin] The token cache is being invalidate because project 21e7b431fb2c49bd8b4b055c796c2816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:02:25.419909 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da75d6b-3212-4d27-90ab-67b9dfa10fcb service swift] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:25.420409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-45925b17-8d8a-41d1-990f-9c53c8ba0495 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:25.421261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-45925b17-8d8a-41d1-990f-9c53c8ba0495 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:25.423461 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1266/2504] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:02:25 2022] DELETE /identity/v3/projects/21e7b431fb2c49bd8b4b055c796c2816 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:25.429026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24191e83-d0a6-4e88-8d9a-3c7d08e9d518 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:25.429367 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24191e83-d0a6-4e88-8d9a-3c7d08e9d518 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:25.430342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24191e83-d0a6-4e88-8d9a-3c7d08e9d518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:25.456872 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-24191e83-d0a6-4e88-8d9a-3c7d08e9d518 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:25.458509 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-24191e83-d0a6-4e88-8d9a-3c7d08e9d518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:25.483162 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:25.521599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd30a81-f83f-44c9-b030-4d86b77c824d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:25.522243 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd30a81-f83f-44c9-b030-4d86b77c824d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:25.522525 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd30a81-f83f-44c9-b030-4d86b77c824d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:25.522801 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd30a81-f83f-44c9-b030-4d86b77c824d admin admin] PATH_INFO: `/v3/users/18f3356426a24e719b138332345799c1` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:25.531861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd30a81-f83f-44c9-b030-4d86b77c824d admin admin] RBAC: Authorizing `identity:delete_user(user_id=18f3356426a24e719b138332345799c1)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:25.535219 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd30a81-f83f-44c9-b030-4d86b77c824d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:25.544165 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1267/2505] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:02:25.552257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c31b7d-c62b-4e21-853f-dd428397f8e2 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:25.552560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c31b7d-c62b-4e21-853f-dd428397f8e2 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:25.552813 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c31b7d-c62b-4e21-853f-dd428397f8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:25.576253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-d3c31b7d-c62b-4e21-853f-dd428397f8e2 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:25.577500 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d3c31b7d-c62b-4e21-853f-dd428397f8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:25.590403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1268/2506] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:25.594773 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c31b7d-c62b-4e21-853f-dd428397f8e2 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:25.608637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c31b7d-c62b-4e21-853f-dd428397f8e2 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:25.613604 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb6df0-ec7a-4004-9abb-399d190d46e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:25.614209 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb6df0-ec7a-4004-9abb-399d190d46e4 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:25.614481 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb6df0-ec7a-4004-9abb-399d190d46e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:25.614747 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb6df0-ec7a-4004-9abb-399d190d46e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:25.615600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb6df0-ec7a-4004-9abb-399d190d46e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:02:25.616045 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb6df0-ec7a-4004-9abb-399d190d46e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:25.618297 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb6df0-ec7a-4004-9abb-399d190d46e4 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:25.638982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1269/2507] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:02:25.639907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb6df0-ec7a-4004-9abb-399d190d46e4 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:25.653172 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb6df0-ec7a-4004-9abb-399d190d46e4 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:25.660982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48657f12-d6e7-4b2b-8552-fb73dd61f668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:25.661585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48657f12-d6e7-4b2b-8552-fb73dd61f668 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:25.661858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48657f12-d6e7-4b2b-8552-fb73dd61f668 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:25.662128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48657f12-d6e7-4b2b-8552-fb73dd61f668 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:25.662879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48657f12-d6e7-4b2b-8552-fb73dd61f668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:25.666140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48657f12-d6e7-4b2b-8552-fb73dd61f668 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:25.713326 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7bd30a81-f83f-44c9-b030-4d86b77c824d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18f3356426a24e719b138332345799c1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:25.724062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7bd30a81-f83f-44c9-b030-4d86b77c824d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18f3356426a24e719b138332345799c1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:25.725395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7bd30a81-f83f-44c9-b030-4d86b77c824d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18f3356426a24e719b138332345799c1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:25.728685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:25.765888 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1270/2508] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:02:25 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:02:25.766736 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48657f12-d6e7-4b2b-8552-fb73dd61f668 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:25.769853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-48657f12-d6e7-4b2b-8552-fb73dd61f668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:25.801314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7bd30a81-f83f-44c9-b030-4d86b77c824d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18f3356426a24e719b138332345799c1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:25.822826 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7bd30a81-f83f-44c9-b030-4d86b77c824d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18f3356426a24e719b138332345799c1'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:25.838110 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1239/2509] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:02:25 2022] DELETE /identity/v3/users/18f3356426a24e719b138332345799c1 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:02:25.840976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f01fa0-d4c6-460e-9b30-6878cf2dae2f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:25.842152 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f01fa0-d4c6-460e-9b30-6878cf2dae2f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:25.842429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f01fa0-d4c6-460e-9b30-6878cf2dae2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:25.884043 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e3f01fa0-d4c6-460e-9b30-6878cf2dae2f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:25.885470 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e3f01fa0-d4c6-460e-9b30-6878cf2dae2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:25.893517 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48657f12-d6e7-4b2b-8552-fb73dd61f668 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:25.989430 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255796b0-c1f0-4898-93a4-6a66dcd96790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:25.989430 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255796b0-c1f0-4898-93a4-6a66dcd96790 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:25.989430 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255796b0-c1f0-4898-93a4-6a66dcd96790 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:25.989430 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255796b0-c1f0-4898-93a4-6a66dcd96790 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:25.990036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255796b0-c1f0-4898-93a4-6a66dcd96790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:25.992385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255796b0-c1f0-4898-93a4-6a66dcd96790 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:25.993315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1240/2510] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:26.006015 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f01fa0-d4c6-460e-9b30-6878cf2dae2f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:26.007596 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e3f01fa0-d4c6-460e-9b30-6878cf2dae2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:26.082929 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f01fa0-d4c6-460e-9b30-6878cf2dae2f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:26.092726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a66d3-bf90-4aeb-93ad-274dbd7b2dc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:26.093405 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a66d3-bf90-4aeb-93ad-274dbd7b2dc6 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:26.093986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a66d3-bf90-4aeb-93ad-274dbd7b2dc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:26.094260 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a66d3-bf90-4aeb-93ad-274dbd7b2dc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:26.098732 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1271/2511] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:02:25 2022] POST /identity/v3/users => generated 292 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:26.100153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255796b0-c1f0-4898-93a4-6a66dcd96790 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:26.109405 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a66d3-bf90-4aeb-93ad-274dbd7b2dc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:26.111053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a66d3-bf90-4aeb-93ad-274dbd7b2dc6 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:26.111702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255796b0-c1f0-4898-93a4-6a66dcd96790 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:26.117694 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff502d-8113-4b11-bc30-9f30f26bbf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:26.118637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff502d-8113-4b11-bc30-9f30f26bbf58 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:26.119209 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff502d-8113-4b11-bc30-9f30f26bbf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:26.119640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff502d-8113-4b11-bc30-9f30f26bbf58 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:26.120463 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff502d-8113-4b11-bc30-9f30f26bbf58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:26.122289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff502d-8113-4b11-bc30-9f30f26bbf58 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:26.124012 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1241/2512] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:26.133780 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1272/2513] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:26.138207 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a66d3-bf90-4aeb-93ad-274dbd7b2dc6 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:26.149550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a66d3-bf90-4aeb-93ad-274dbd7b2dc6 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:26.154858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472813e-914d-42d3-ada5-97b45b5118cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:26.155572 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472813e-914d-42d3-ada5-97b45b5118cb admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:26.155921 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472813e-914d-42d3-ada5-97b45b5118cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:26.156259 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472813e-914d-42d3-ada5-97b45b5118cb admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:26.156969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472813e-914d-42d3-ada5-97b45b5118cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:26.158795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472813e-914d-42d3-ada5-97b45b5118cb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:26.174696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff502d-8113-4b11-bc30-9f30f26bbf58 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:26.181896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1242/2514] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:26.185594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff502d-8113-4b11-bc30-9f30f26bbf58 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:26.199530 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f21a4c-c4b6-4fda-9ff7-0e34cc6e56ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:26.201071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f21a4c-c4b6-4fda-9ff7-0e34cc6e56ed admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:26.201422 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f21a4c-c4b6-4fda-9ff7-0e34cc6e56ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:26.201766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f21a4c-c4b6-4fda-9ff7-0e34cc6e56ed admin admin] PATH_INFO: `/v3/projects/aa0df72533d34237a9b1868391740224/users/2bc1bd791342420c99321b3ebd73bd57/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:26.206252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:26.228704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f21a4c-c4b6-4fda-9ff7-0e34cc6e56ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa0df72533d34237a9b1868391740224, user_id=2bc1bd791342420c99321b3ebd73bd57, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:26.232115 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f21a4c-c4b6-4fda-9ff7-0e34cc6e56ed admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:26.334142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1273/2515] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:26 2022] PUT /identity/v3/projects/aa0df72533d34237a9b1868391740224/users/2bc1bd791342420c99321b3ebd73bd57/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:02:26.339959 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ccc0f-cbb1-4fc7-bcd2-5c6bc0d0976d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:26.340234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ccc0f-cbb1-4fc7-bcd2-5c6bc0d0976d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:26.340759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ccc0f-cbb1-4fc7-bcd2-5c6bc0d0976d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:26.356954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f21a4c-c4b6-4fda-9ff7-0e34cc6e56ed admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:26.359170 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-69f21a4c-c4b6-4fda-9ff7-0e34cc6e56ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:26.400264 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-295ccc0f-cbb1-4fc7-bcd2-5c6bc0d0976d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:26.404277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-295ccc0f-cbb1-4fc7-bcd2-5c6bc0d0976d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:26.494600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f21a4c-c4b6-4fda-9ff7-0e34cc6e56ed admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:26.512374 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821a75c-34f4-4f62-b460-e894683dfc7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:26.513022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821a75c-34f4-4f62-b460-e894683dfc7f service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:26.513264 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821a75c-34f4-4f62-b460-e894683dfc7f service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:26.513534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821a75c-34f4-4f62-b460-e894683dfc7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:26.518228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7821a75c-34f4-4f62-b460-e894683dfc7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:26.545823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821a75c-34f4-4f62-b460-e894683dfc7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:26.549544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821a75c-34f4-4f62-b460-e894683dfc7f service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:26.559930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1243/2516] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:26.569205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ccc0f-cbb1-4fc7-bcd2-5c6bc0d0976d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:26.587311 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1274/2517] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:02:26.598659 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:26.661298 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ccc0f-cbb1-4fc7-bcd2-5c6bc0d0976d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:26.666510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7389ce4-d815-4bdb-99c3-8e72c6451a01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:26.667202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7389ce4-d815-4bdb-99c3-8e72c6451a01 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:26.667472 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7389ce4-d815-4bdb-99c3-8e72c6451a01 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:26.667743 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7389ce4-d815-4bdb-99c3-8e72c6451a01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:26.681002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7389ce4-d815-4bdb-99c3-8e72c6451a01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:26.682464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7389ce4-d815-4bdb-99c3-8e72c6451a01 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:26.693124 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1244/2518] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:26.708435 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821a75c-34f4-4f62-b460-e894683dfc7f service nova] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:26.709786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7821a75c-34f4-4f62-b460-e894683dfc7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:26.731118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821a75c-34f4-4f62-b460-e894683dfc7f service nova] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:26.746611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01a4a0-b848-4230-85d4-6965c38087aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:26.747632 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01a4a0-b848-4230-85d4-6965c38087aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:26.747988 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01a4a0-b848-4230-85d4-6965c38087aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:26.748321 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01a4a0-b848-4230-85d4-6965c38087aa admin admin] PATH_INFO: `/v3/projects/1181cb0087af4eaea5b135bca726725e` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:26.753386 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01a4a0-b848-4230-85d4-6965c38087aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1181cb0087af4eaea5b135bca726725e)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:26.753386 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01a4a0-b848-4230-85d4-6965c38087aa admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:26.926894 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0a01a4a0-b848-4230-85d4-6965c38087aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1181cb0087af4eaea5b135bca726725e', 'request_id': 'req-0a01a4a0-b848-4230-85d4-6965c38087aa'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:26.935076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0a01a4a0-b848-4230-85d4-6965c38087aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1181cb0087af4eaea5b135bca726725e', 'request_id': 'req-0a01a4a0-b848-4230-85d4-6965c38087aa'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:27.035711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0a01a4a0-b848-4230-85d4-6965c38087aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1181cb0087af4eaea5b135bca726725e', 'request_id': 'req-0a01a4a0-b848-4230-85d4-6965c38087aa'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:27.059329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0a01a4a0-b848-4230-85d4-6965c38087aa admin admin] The token cache is being invalidate because project 1181cb0087af4eaea5b135bca726725e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:02:27.059329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0a01a4a0-b848-4230-85d4-6965c38087aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:27.061011 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0a01a4a0-b848-4230-85d4-6965c38087aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:27.066036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1275/2519] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:02:26 2022] DELETE /identity/v3/projects/1181cb0087af4eaea5b135bca726725e => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:02:27.069996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7389ce4-d815-4bdb-99c3-8e72c6451a01 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:27.086938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7389ce4-d815-4bdb-99c3-8e72c6451a01 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:27.198734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5c59a-6d3f-471f-ac21-cd0fd78fff1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:27.199685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5c59a-6d3f-471f-ac21-cd0fd78fff1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:27.200074 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5c59a-6d3f-471f-ac21-cd0fd78fff1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:27.200622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5c59a-6d3f-471f-ac21-cd0fd78fff1a admin admin] PATH_INFO: `/v3/users/575f0abf57694440b407907908bf9841` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:27.202381 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5c59a-6d3f-471f-ac21-cd0fd78fff1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=575f0abf57694440b407907908bf9841)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:27.206097 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5c59a-6d3f-471f-ac21-cd0fd78fff1a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:27.479522 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cfc5c59a-6d3f-471f-ac21-cd0fd78fff1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '575f0abf57694440b407907908bf9841'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:27.487838 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cfc5c59a-6d3f-471f-ac21-cd0fd78fff1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '575f0abf57694440b407907908bf9841'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:27.488688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cfc5c59a-6d3f-471f-ac21-cd0fd78fff1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '575f0abf57694440b407907908bf9841'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:27.560883 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cfc5c59a-6d3f-471f-ac21-cd0fd78fff1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '575f0abf57694440b407907908bf9841'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:27.576434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cfc5c59a-6d3f-471f-ac21-cd0fd78fff1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '575f0abf57694440b407907908bf9841'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:27.583459 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1245/2520] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:02:27 2022] DELETE /identity/v3/users/575f0abf57694440b407907908bf9841 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:02:28.242567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01a4a0-b848-4230-85d4-6965c38087aa admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:28.245459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0a01a4a0-b848-4230-85d4-6965c38087aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:28.281191 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01a4a0-b848-4230-85d4-6965c38087aa admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:28.399509 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:28.421008 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d995cb-83db-4c97-9bda-ff4f8726d4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:28.421977 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d995cb-83db-4c97-9bda-ff4f8726d4f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:28.422358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d995cb-83db-4c97-9bda-ff4f8726d4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:28.422970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d995cb-83db-4c97-9bda-ff4f8726d4f4 admin admin] PATH_INFO: `/v3/projects/4e0fe5e0b76048059ea58db6921661f1` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:28.424305 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d995cb-83db-4c97-9bda-ff4f8726d4f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e0fe5e0b76048059ea58db6921661f1)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:28.440947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d995cb-83db-4c97-9bda-ff4f8726d4f4 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:28.700145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-19d995cb-83db-4c97-9bda-ff4f8726d4f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e0fe5e0b76048059ea58db6921661f1', 'request_id': 'req-19d995cb-83db-4c97-9bda-ff4f8726d4f4'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:28.716341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-19d995cb-83db-4c97-9bda-ff4f8726d4f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e0fe5e0b76048059ea58db6921661f1', 'request_id': 'req-19d995cb-83db-4c97-9bda-ff4f8726d4f4'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:28.798177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-19d995cb-83db-4c97-9bda-ff4f8726d4f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e0fe5e0b76048059ea58db6921661f1', 'request_id': 'req-19d995cb-83db-4c97-9bda-ff4f8726d4f4'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:28.815066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-19d995cb-83db-4c97-9bda-ff4f8726d4f4 admin admin] The token cache is being invalidate because project 4e0fe5e0b76048059ea58db6921661f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:02:28.815646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-19d995cb-83db-4c97-9bda-ff4f8726d4f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:28.818204 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-19d995cb-83db-4c97-9bda-ff4f8726d4f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:28.820665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1276/2521] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:02:28 2022] DELETE /identity/v3/projects/4e0fe5e0b76048059ea58db6921661f1 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:28.826061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8ef08-7875-461e-987a-e0c80393c872 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:28.826576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8ef08-7875-461e-987a-e0c80393c872 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:28.826865 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8ef08-7875-461e-987a-e0c80393c872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:28.852524 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-33b8ef08-7875-461e-987a-e0c80393c872 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:28.854375 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-33b8ef08-7875-461e-987a-e0c80393c872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:28.882488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:28.925767 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1246/2522] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:28.935446 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920ac83-bb2d-42fe-92ea-5f3e90404c46 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:28.936600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920ac83-bb2d-42fe-92ea-5f3e90404c46 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:28.937482 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920ac83-bb2d-42fe-92ea-5f3e90404c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:28.977476 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e920ac83-bb2d-42fe-92ea-5f3e90404c46 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:28.978711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e920ac83-bb2d-42fe-92ea-5f3e90404c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:28.992320 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8ef08-7875-461e-987a-e0c80393c872 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:28.997831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1277/2523] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:29.006575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8ef08-7875-461e-987a-e0c80393c872 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:29.016739 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24cff6-ff64-4f34-8b70-1107ab8834ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:29.017520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24cff6-ff64-4f34-8b70-1107ab8834ee admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:29.017906 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24cff6-ff64-4f34-8b70-1107ab8834ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:29.019031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24cff6-ff64-4f34-8b70-1107ab8834ee admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:29.019942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24cff6-ff64-4f34-8b70-1107ab8834ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:02:29.019942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24cff6-ff64-4f34-8b70-1107ab8834ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:29.022348 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24cff6-ff64-4f34-8b70-1107ab8834ee admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:29.051973 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1247/2524] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:02:29.052676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920ac83-bb2d-42fe-92ea-5f3e90404c46 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:29.066474 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920ac83-bb2d-42fe-92ea-5f3e90404c46 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:29.074662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06aa21-379e-4034-8de6-07e1f3a15bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:29.076227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06aa21-379e-4034-8de6-07e1f3a15bdc admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:29.076678 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06aa21-379e-4034-8de6-07e1f3a15bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:29.077824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06aa21-379e-4034-8de6-07e1f3a15bdc admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:29.079040 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06aa21-379e-4034-8de6-07e1f3a15bdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:29.081126 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06aa21-379e-4034-8de6-07e1f3a15bdc admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:29.189904 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1278/2525] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:02:29 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:02:29.196760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24cff6-ff64-4f34-8b70-1107ab8834ee admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:29.210737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24cff6-ff64-4f34-8b70-1107ab8834ee admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:29.288655 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1779507-c9ed-4db5-82a8-c7958cbbc8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:29.289202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1779507-c9ed-4db5-82a8-c7958cbbc8a7 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:29.289538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1779507-c9ed-4db5-82a8-c7958cbbc8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:29.290436 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1779507-c9ed-4db5-82a8-c7958cbbc8a7 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:29.291086 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1779507-c9ed-4db5-82a8-c7958cbbc8a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:29.293512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1779507-c9ed-4db5-82a8-c7958cbbc8a7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:29.419340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1248/2526] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:02:29 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:29.424292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06aa21-379e-4034-8de6-07e1f3a15bdc admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:29.443736 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06aa21-379e-4034-8de6-07e1f3a15bdc admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:29.453944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f50140-0105-45a8-9e9a-710d29583bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:29.455497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f50140-0105-45a8-9e9a-710d29583bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:29.455941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f50140-0105-45a8-9e9a-710d29583bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:29.457789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f50140-0105-45a8-9e9a-710d29583bd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:29.458933 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f50140-0105-45a8-9e9a-710d29583bd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:29.463550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f50140-0105-45a8-9e9a-710d29583bd1 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:29.483997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1779507-c9ed-4db5-82a8-c7958cbbc8a7 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:29.492531 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1279/2527] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:29.497620 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1779507-c9ed-4db5-82a8-c7958cbbc8a7 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:29.505586 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750daa5-e455-4231-85d2-41ac524d8373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:29.507426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750daa5-e455-4231-85d2-41ac524d8373 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:29.507426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750daa5-e455-4231-85d2-41ac524d8373 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:29.507426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750daa5-e455-4231-85d2-41ac524d8373 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:29.507426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750daa5-e455-4231-85d2-41ac524d8373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:29.512792 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750daa5-e455-4231-85d2-41ac524d8373 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:29.537183 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1249/2528] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:02:29.537612 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f50140-0105-45a8-9e9a-710d29583bd1 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:29.553475 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f50140-0105-45a8-9e9a-710d29583bd1 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:29.564179 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b9e67-f68b-40c1-83c5-dcf78294a97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:29.565169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b9e67-f68b-40c1-83c5-dcf78294a97a admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:29.567505 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b9e67-f68b-40c1-83c5-dcf78294a97a admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:29.567505 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b9e67-f68b-40c1-83c5-dcf78294a97a admin admin] PATH_INFO: `/v3/projects/327450e6a5ed456faacdb3e2dfed2d6d/users/04c8e27dd63342a69222a3bdfc101dc5/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:29.593222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b9e67-f68b-40c1-83c5-dcf78294a97a admin admin] RBAC: Authorizing `identity:create_grant(project_id=327450e6a5ed456faacdb3e2dfed2d6d, user_id=04c8e27dd63342a69222a3bdfc101dc5, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:29.596785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b9e67-f68b-40c1-83c5-dcf78294a97a admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:29.692941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1280/2529] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:29 2022] PUT /identity/v3/projects/327450e6a5ed456faacdb3e2dfed2d6d/users/04c8e27dd63342a69222a3bdfc101dc5/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:29.704796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45c92d-7b97-4852-a392-70545aa33d6e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:29.705193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45c92d-7b97-4852-a392-70545aa33d6e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:29.705369 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45c92d-7b97-4852-a392-70545aa33d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:29.763925 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-7f45c92d-7b97-4852-a392-70545aa33d6e None None] MFA Rules not processed for user `04c8e27dd63342a69222a3bdfc101dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:29.773667 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7f45c92d-7b97-4852-a392-70545aa33d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:30.128170 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1250/2530] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:30.129360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b9e67-f68b-40c1-83c5-dcf78294a97a admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:30.228166 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b9e67-f68b-40c1-83c5-dcf78294a97a admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:30.257940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832324b3-e374-4f9a-b1c7-84ad056645b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:30.258630 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832324b3-e374-4f9a-b1c7-84ad056645b6 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:30.258913 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832324b3-e374-4f9a-b1c7-84ad056645b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:30.259264 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832324b3-e374-4f9a-b1c7-84ad056645b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:30.259918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832324b3-e374-4f9a-b1c7-84ad056645b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:30.266903 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832324b3-e374-4f9a-b1c7-84ad056645b6 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:30.420638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45c92d-7b97-4852-a392-70545aa33d6e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:30.420899 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1281/2531] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:02:30 2022] POST /identity/v3/projects => generated 387 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:30.449586 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45c92d-7b97-4852-a392-70545aa33d6e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:30.493613 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:30.569284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb0366-0052-480a-9ec9-e2f0a26703de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:30.569896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb0366-0052-480a-9ec9-e2f0a26703de admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:30.570169 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb0366-0052-480a-9ec9-e2f0a26703de admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:30.570436 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb0366-0052-480a-9ec9-e2f0a26703de admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:30.573842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb0366-0052-480a-9ec9-e2f0a26703de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:30.576357 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb0366-0052-480a-9ec9-e2f0a26703de admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:30.705060 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1251/2532] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:02:30 2022] POST /identity/v3/users => generated 300 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:30.709693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832324b3-e374-4f9a-b1c7-84ad056645b6 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:30.720995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:30.743958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832324b3-e374-4f9a-b1c7-84ad056645b6 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:30.752136 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762aaa92-ec9d-44f7-954d-a17c4367ce21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:30.753101 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762aaa92-ec9d-44f7-954d-a17c4367ce21 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:30.753484 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762aaa92-ec9d-44f7-954d-a17c4367ce21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:30.754452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762aaa92-ec9d-44f7-954d-a17c4367ce21 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:30.755407 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762aaa92-ec9d-44f7-954d-a17c4367ce21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:30.760306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762aaa92-ec9d-44f7-954d-a17c4367ce21 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:30.777091 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1282/2533] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:30.777549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb0366-0052-480a-9ec9-e2f0a26703de admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:30.788940 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb0366-0052-480a-9ec9-e2f0a26703de admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:30.798108 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71001cb-dfcc-4192-ae36-8d36d67c48a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:30.799589 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71001cb-dfcc-4192-ae36-8d36d67c48a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:30.799877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71001cb-dfcc-4192-ae36-8d36d67c48a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:30.800189 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71001cb-dfcc-4192-ae36-8d36d67c48a1 admin admin] PATH_INFO: `/v3/projects/9c205f8439e44e108c9ee2adb39cff0d/users/8b88ccc4e7d546aab8f0ffd7b47c2e45/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:30.829939 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71001cb-dfcc-4192-ae36-8d36d67c48a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c205f8439e44e108c9ee2adb39cff0d, user_id=8b88ccc4e7d546aab8f0ffd7b47c2e45, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:30.839948 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71001cb-dfcc-4192-ae36-8d36d67c48a1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:30.930479 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1252/2534] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:30 2022] PUT /identity/v3/projects/9c205f8439e44e108c9ee2adb39cff0d/users/8b88ccc4e7d546aab8f0ffd7b47c2e45/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:30.935159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762aaa92-ec9d-44f7-954d-a17c4367ce21 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:30.950171 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762aaa92-ec9d-44f7-954d-a17c4367ce21 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:31.094076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e43f76-7691-4b17-97e3-52e9470026d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:31.094706 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e43f76-7691-4b17-97e3-52e9470026d1 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:31.094987 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e43f76-7691-4b17-97e3-52e9470026d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:31.095296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e43f76-7691-4b17-97e3-52e9470026d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:31.095945 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e43f76-7691-4b17-97e3-52e9470026d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:31.097549 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e43f76-7691-4b17-97e3-52e9470026d1 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:31.124532 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1283/2535] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:31.128432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71001cb-dfcc-4192-ae36-8d36d67c48a1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:31.164943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71001cb-dfcc-4192-ae36-8d36d67c48a1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:31.181868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c91cd1-2909-419f-9cd1-9fafc0f0a800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:31.183008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c91cd1-2909-419f-9cd1-9fafc0f0a800 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:31.184982 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c91cd1-2909-419f-9cd1-9fafc0f0a800 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:31.185544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c91cd1-2909-419f-9cd1-9fafc0f0a800 admin admin] PATH_INFO: `/v3/projects/9c205f8439e44e108c9ee2adb39cff0d/users/8b88ccc4e7d546aab8f0ffd7b47c2e45/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:31.190847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c91cd1-2909-419f-9cd1-9fafc0f0a800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c205f8439e44e108c9ee2adb39cff0d, user_id=8b88ccc4e7d546aab8f0ffd7b47c2e45, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:31.198567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c91cd1-2909-419f-9cd1-9fafc0f0a800 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:31.221770 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1253/2536] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:31 2022] PUT /identity/v3/projects/9c205f8439e44e108c9ee2adb39cff0d/users/8b88ccc4e7d546aab8f0ffd7b47c2e45/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:31.232398 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bca39e-35b5-4a87-92cf-c42a17aca802 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:31.232920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bca39e-35b5-4a87-92cf-c42a17aca802 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:31.233508 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bca39e-35b5-4a87-92cf-c42a17aca802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:31.286656 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-69bca39e-35b5-4a87-92cf-c42a17aca802 None None] MFA Rules not processed for user `8b88ccc4e7d546aab8f0ffd7b47c2e45`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:31.287990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-69bca39e-35b5-4a87-92cf-c42a17aca802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:31.439550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1284/2537] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:31.453825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c91cd1-2909-419f-9cd1-9fafc0f0a800 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:31.456226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-77c91cd1-2909-419f-9cd1-9fafc0f0a800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:31.583916 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c91cd1-2909-419f-9cd1-9fafc0f0a800 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:31.596685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83394bd-720b-4843-a422-00c91d042d54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:31.597386 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83394bd-720b-4843-a422-00c91d042d54 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:31.597739 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83394bd-720b-4843-a422-00c91d042d54 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:31.598078 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83394bd-720b-4843-a422-00c91d042d54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:31.618321 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83394bd-720b-4843-a422-00c91d042d54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:31.622288 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83394bd-720b-4843-a422-00c91d042d54 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:31.645081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1254/2538] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:31.668106 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bca39e-35b5-4a87-92cf-c42a17aca802 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:31.826954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bca39e-35b5-4a87-92cf-c42a17aca802 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:31.840191 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c710b73-9797-40e1-8f64-a0b4af2e6702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:31.840523 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c710b73-9797-40e1-8f64-a0b4af2e6702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:31.840749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c710b73-9797-40e1-8f64-a0b4af2e6702 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:31.841023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c710b73-9797-40e1-8f64-a0b4af2e6702 admin admin] PATH_INFO: `/v3/users/04c8e27dd63342a69222a3bdfc101dc5` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:31.845256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c710b73-9797-40e1-8f64-a0b4af2e6702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04c8e27dd63342a69222a3bdfc101dc5)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:31.847217 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c710b73-9797-40e1-8f64-a0b4af2e6702 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:31.876550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:32.129295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9c710b73-9797-40e1-8f64-a0b4af2e6702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04c8e27dd63342a69222a3bdfc101dc5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:32.144765 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9c710b73-9797-40e1-8f64-a0b4af2e6702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04c8e27dd63342a69222a3bdfc101dc5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:32.190927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9c710b73-9797-40e1-8f64-a0b4af2e6702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04c8e27dd63342a69222a3bdfc101dc5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:32.192211 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9c710b73-9797-40e1-8f64-a0b4af2e6702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04c8e27dd63342a69222a3bdfc101dc5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:32.193118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-9c710b73-9797-40e1-8f64-a0b4af2e6702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04c8e27dd63342a69222a3bdfc101dc5'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:32.694889 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1285/2539] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:02:31 2022] DELETE /identity/v3/users/04c8e27dd63342a69222a3bdfc101dc5 => generated 0 bytes in 1029 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:32.700536 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad8314-a297-4b9d-b2dc-9ed1c3dd637d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:32.701075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad8314-a297-4b9d-b2dc-9ed1c3dd637d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:32.701183 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad8314-a297-4b9d-b2dc-9ed1c3dd637d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:32.719932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5fad8314-a297-4b9d-b2dc-9ed1c3dd637d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:32.721217 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5fad8314-a297-4b9d-b2dc-9ed1c3dd637d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:32.788056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1255/2540] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:32.798302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c710b73-9797-40e1-8f64-a0b4af2e6702 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:32.800017 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9c710b73-9797-40e1-8f64-a0b4af2e6702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:32.871750 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c710b73-9797-40e1-8f64-a0b4af2e6702 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:32.876254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6a6ae-7f1b-4450-89bb-0a51d647b82e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:32.877030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6a6ae-7f1b-4450-89bb-0a51d647b82e service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:32.877384 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6a6ae-7f1b-4450-89bb-0a51d647b82e service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:32.877773 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6a6ae-7f1b-4450-89bb-0a51d647b82e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:32.893275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6a6ae-7f1b-4450-89bb-0a51d647b82e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:32.895047 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6a6ae-7f1b-4450-89bb-0a51d647b82e service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:32.919254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1286/2541] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:33.626121 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad8314-a297-4b9d-b2dc-9ed1c3dd637d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:33.627727 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5fad8314-a297-4b9d-b2dc-9ed1c3dd637d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:33.650274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad8314-a297-4b9d-b2dc-9ed1c3dd637d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:33.663958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd0585-3a24-4922-9172-8ccc96c8252b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:33.665421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd0585-3a24-4922-9172-8ccc96c8252b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:33.669760 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd0585-3a24-4922-9172-8ccc96c8252b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:33.670407 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd0585-3a24-4922-9172-8ccc96c8252b admin admin] PATH_INFO: `/v3/projects/327450e6a5ed456faacdb3e2dfed2d6d` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:33.671885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd0585-3a24-4922-9172-8ccc96c8252b admin admin] RBAC: Authorizing `identity:delete_project(project_id=327450e6a5ed456faacdb3e2dfed2d6d)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:33.674069 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd0585-3a24-4922-9172-8ccc96c8252b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:33.712129 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:33.897048 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ffcd0585-3a24-4922-9172-8ccc96c8252b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '327450e6a5ed456faacdb3e2dfed2d6d', 'request_id': 'req-ffcd0585-3a24-4922-9172-8ccc96c8252b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:33.907308 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ffcd0585-3a24-4922-9172-8ccc96c8252b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '327450e6a5ed456faacdb3e2dfed2d6d', 'request_id': 'req-ffcd0585-3a24-4922-9172-8ccc96c8252b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:33.915831 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ffcd0585-3a24-4922-9172-8ccc96c8252b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '327450e6a5ed456faacdb3e2dfed2d6d', 'request_id': 'req-ffcd0585-3a24-4922-9172-8ccc96c8252b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:33.983616 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ffcd0585-3a24-4922-9172-8ccc96c8252b admin admin] The token cache is being invalidate because project 327450e6a5ed456faacdb3e2dfed2d6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:02:33.987971 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ffcd0585-3a24-4922-9172-8ccc96c8252b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:33.987971 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-ffcd0585-3a24-4922-9172-8ccc96c8252b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:33.992727 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1256/2542] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:02:33 2022] DELETE /identity/v3/projects/327450e6a5ed456faacdb3e2dfed2d6d => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:33.994556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6a6ae-7f1b-4450-89bb-0a51d647b82e service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:34.015827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6a6ae-7f1b-4450-89bb-0a51d647b82e service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:34.182128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c190d4b-63eb-4792-a63d-d9bb3771adb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:34.182980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c190d4b-63eb-4792-a63d-d9bb3771adb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:34.183303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c190d4b-63eb-4792-a63d-d9bb3771adb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:34.190400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c190d4b-63eb-4792-a63d-d9bb3771adb7 admin admin] PATH_INFO: `/v3/users/8b88ccc4e7d546aab8f0ffd7b47c2e45` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:34.194038 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c190d4b-63eb-4792-a63d-d9bb3771adb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b88ccc4e7d546aab8f0ffd7b47c2e45)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:34.197342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c190d4b-63eb-4792-a63d-d9bb3771adb7 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:34.442610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:34.464441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1c190d4b-63eb-4792-a63d-d9bb3771adb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b88ccc4e7d546aab8f0ffd7b47c2e45'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:34.475176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1c190d4b-63eb-4792-a63d-d9bb3771adb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b88ccc4e7d546aab8f0ffd7b47c2e45'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:34.500119 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608342f-3205-4aad-9eaa-3d5c3dfec65c None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:34.500469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608342f-3205-4aad-9eaa-3d5c3dfec65c None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:34.500757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608342f-3205-4aad-9eaa-3d5c3dfec65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:34.501067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1c190d4b-63eb-4792-a63d-d9bb3771adb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b88ccc4e7d546aab8f0ffd7b47c2e45'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:34.502731 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1c190d4b-63eb-4792-a63d-d9bb3771adb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b88ccc4e7d546aab8f0ffd7b47c2e45'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:34.503574 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1c190d4b-63eb-4792-a63d-d9bb3771adb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b88ccc4e7d546aab8f0ffd7b47c2e45'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:34.547894 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6608342f-3205-4aad-9eaa-3d5c3dfec65c None None] MFA Rules not processed for user `2bc1bd791342420c99321b3ebd73bd57`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:34.549159 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6608342f-3205-4aad-9eaa-3d5c3dfec65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:34.598981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1287/2543] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:02:33 2022] DELETE /identity/v3/users/8b88ccc4e7d546aab8f0ffd7b47c2e45 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:34.697208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1257/2544] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:34.708641 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c190d4b-63eb-4792-a63d-d9bb3771adb7 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:34.710985 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1c190d4b-63eb-4792-a63d-d9bb3771adb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:34.836437 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c190d4b-63eb-4792-a63d-d9bb3771adb7 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:34.847050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f19fa4-0f56-4f83-a8d8-fe22f7dce4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:34.847834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f19fa4-0f56-4f83-a8d8-fe22f7dce4f5 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:34.848249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f19fa4-0f56-4f83-a8d8-fe22f7dce4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:34.848536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f19fa4-0f56-4f83-a8d8-fe22f7dce4f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:34.849160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f19fa4-0f56-4f83-a8d8-fe22f7dce4f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:34.853326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f19fa4-0f56-4f83-a8d8-fe22f7dce4f5 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:34.979749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1288/2545] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:02:34 2022] POST /identity/v3/projects => generated 385 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:02:34.984632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608342f-3205-4aad-9eaa-3d5c3dfec65c None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:34.999021 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608342f-3205-4aad-9eaa-3d5c3dfec65c None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:35.134083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdc0d2-4682-4f55-9120-4acb32129e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:35.134896 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdc0d2-4682-4f55-9120-4acb32129e50 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:35.135597 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdc0d2-4682-4f55-9120-4acb32129e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:35.136070 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdc0d2-4682-4f55-9120-4acb32129e50 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:35.137001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdc0d2-4682-4f55-9120-4acb32129e50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:35.139353 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdc0d2-4682-4f55-9120-4acb32129e50 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:35.245617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1258/2546] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:02:34 2022] POST /identity/v3/users => generated 299 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:35.250451 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f19fa4-0f56-4f83-a8d8-fe22f7dce4f5 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:35.264400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f19fa4-0f56-4f83-a8d8-fe22f7dce4f5 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:35.285265 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de3add-bc27-464a-a3b0-14f72b9db8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:35.286325 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de3add-bc27-464a-a3b0-14f72b9db8b5 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:35.286762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de3add-bc27-464a-a3b0-14f72b9db8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:35.287033 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de3add-bc27-464a-a3b0-14f72b9db8b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:35.287853 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de3add-bc27-464a-a3b0-14f72b9db8b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:35.290006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de3add-bc27-464a-a3b0-14f72b9db8b5 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:35.305002 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1289/2547] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:35.306430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdc0d2-4682-4f55-9120-4acb32129e50 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:35.317578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdc0d2-4682-4f55-9120-4acb32129e50 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:35.322909 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d43531-4a1f-4a26-8928-aa203bcefb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:35.323642 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d43531-4a1f-4a26-8928-aa203bcefb2d admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:35.323932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d43531-4a1f-4a26-8928-aa203bcefb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:35.324284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d43531-4a1f-4a26-8928-aa203bcefb2d admin admin] PATH_INFO: `/v3/projects/dcca2432532f4d08b391afbb7f15cf6a/users/643bb62a05e64c6c844fda8ec2ee7e41/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:35.347666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d43531-4a1f-4a26-8928-aa203bcefb2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcca2432532f4d08b391afbb7f15cf6a, user_id=643bb62a05e64c6c844fda8ec2ee7e41, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:35.351034 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d43531-4a1f-4a26-8928-aa203bcefb2d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:35.426630 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de3add-bc27-464a-a3b0-14f72b9db8b5 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:35.428835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-22de3add-bc27-464a-a3b0-14f72b9db8b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:35.446190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:35.450076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de3add-bc27-464a-a3b0-14f72b9db8b5 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:35.473481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1259/2548] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:35 2022] PUT /identity/v3/projects/dcca2432532f4d08b391afbb7f15cf6a/users/643bb62a05e64c6c844fda8ec2ee7e41/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:02:35.475368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d43531-4a1f-4a26-8928-aa203bcefb2d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:35.555360 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a64c69-6a91-41d6-a509-066acd35663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:35.559974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a64c69-6a91-41d6-a509-066acd35663b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:35.560393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a64c69-6a91-41d6-a509-066acd35663b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:35.560939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a64c69-6a91-41d6-a509-066acd35663b admin admin] PATH_INFO: `/v3/projects/9c205f8439e44e108c9ee2adb39cff0d` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:35.562981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a64c69-6a91-41d6-a509-066acd35663b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c205f8439e44e108c9ee2adb39cff0d)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:35.565645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a64c69-6a91-41d6-a509-066acd35663b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:35.623562 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d43531-4a1f-4a26-8928-aa203bcefb2d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:35.630086 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f394cd-bd14-429c-9061-36a6767f1aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:35.630799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f394cd-bd14-429c-9061-36a6767f1aac admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:35.631083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f394cd-bd14-429c-9061-36a6767f1aac admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:35.631383 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f394cd-bd14-429c-9061-36a6767f1aac admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:35.632042 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f394cd-bd14-429c-9061-36a6767f1aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:35.634961 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f394cd-bd14-429c-9061-36a6767f1aac admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:35.659314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1260/2549] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:02:35.661922 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f394cd-bd14-429c-9061-36a6767f1aac admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:35.679040 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f394cd-bd14-429c-9061-36a6767f1aac admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:35.685230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b4686-93c7-47ff-9ed9-56906732d794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:35.685939 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b4686-93c7-47ff-9ed9-56906732d794 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:35.686231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b4686-93c7-47ff-9ed9-56906732d794 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:35.686517 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b4686-93c7-47ff-9ed9-56906732d794 admin admin] PATH_INFO: `/v3/projects/dcca2432532f4d08b391afbb7f15cf6a/users/643bb62a05e64c6c844fda8ec2ee7e41/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:35.692166 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b4686-93c7-47ff-9ed9-56906732d794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcca2432532f4d08b391afbb7f15cf6a, user_id=643bb62a05e64c6c844fda8ec2ee7e41, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:35.697207 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b4686-93c7-47ff-9ed9-56906732d794 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:35.700274 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:35.711958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1261/2550] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:35 2022] PUT /identity/v3/projects/dcca2432532f4d08b391afbb7f15cf6a/users/643bb62a05e64c6c844fda8ec2ee7e41/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:35.756993 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-07a64c69-6a91-41d6-a509-066acd35663b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c205f8439e44e108c9ee2adb39cff0d', 'request_id': 'req-07a64c69-6a91-41d6-a509-066acd35663b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:35.765648 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-07a64c69-6a91-41d6-a509-066acd35663b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c205f8439e44e108c9ee2adb39cff0d', 'request_id': 'req-07a64c69-6a91-41d6-a509-066acd35663b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:35.846375 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-07a64c69-6a91-41d6-a509-066acd35663b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c205f8439e44e108c9ee2adb39cff0d', 'request_id': 'req-07a64c69-6a91-41d6-a509-066acd35663b'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:35.874176 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-07a64c69-6a91-41d6-a509-066acd35663b admin admin] The token cache is being invalidate because project 9c205f8439e44e108c9ee2adb39cff0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:02:35.874518 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-07a64c69-6a91-41d6-a509-066acd35663b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:35.875273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-07a64c69-6a91-41d6-a509-066acd35663b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:35.878195 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1290/2551] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:02:35 2022] DELETE /identity/v3/projects/9c205f8439e44e108c9ee2adb39cff0d => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:35.884768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2febc8-642c-444e-9dbf-2b08d89ac2f5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:35.885103 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2febc8-642c-444e-9dbf-2b08d89ac2f5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:35.885370 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2febc8-642c-444e-9dbf-2b08d89ac2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:35.917474 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3e2febc8-642c-444e-9dbf-2b08d89ac2f5 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:35.919923 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3e2febc8-642c-444e-9dbf-2b08d89ac2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:36.022284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1262/2552] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:36.030810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db51c03-2038-4989-ba4d-6f63bbefef81 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:36.031090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db51c03-2038-4989-ba4d-6f63bbefef81 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:36.031390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db51c03-2038-4989-ba4d-6f63bbefef81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:36.067935 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-2db51c03-2038-4989-ba4d-6f63bbefef81 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:36.070687 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-2db51c03-2038-4989-ba4d-6f63bbefef81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:36.155864 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1291/2553] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:36.159915 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2febc8-642c-444e-9dbf-2b08d89ac2f5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:36.172240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2febc8-642c-444e-9dbf-2b08d89ac2f5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:36.178272 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7d8f3-3c4d-4a79-93b9-3ac3eddfc66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:36.179138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7d8f3-3c4d-4a79-93b9-3ac3eddfc66e admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:36.179576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7d8f3-3c4d-4a79-93b9-3ac3eddfc66e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:36.180627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7d8f3-3c4d-4a79-93b9-3ac3eddfc66e admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:36.181654 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7d8f3-3c4d-4a79-93b9-3ac3eddfc66e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:02:36.182156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7d8f3-3c4d-4a79-93b9-3ac3eddfc66e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:36.184937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7d8f3-3c4d-4a79-93b9-3ac3eddfc66e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:36.200136 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1263/2554] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:02:36.204802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db51c03-2038-4989-ba4d-6f63bbefef81 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:36.216933 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db51c03-2038-4989-ba4d-6f63bbefef81 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:36.222428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea06a33-472c-43e8-86bf-34995affe1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:36.223287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea06a33-472c-43e8-86bf-34995affe1db admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:36.223738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea06a33-472c-43e8-86bf-34995affe1db admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:36.224746 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea06a33-472c-43e8-86bf-34995affe1db admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:36.225714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea06a33-472c-43e8-86bf-34995affe1db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:36.228188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea06a33-472c-43e8-86bf-34995affe1db admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:36.362832 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1292/2555] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:02:36 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:02:36.368791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7d8f3-3c4d-4a79-93b9-3ac3eddfc66e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:36.384686 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7d8f3-3c4d-4a79-93b9-3ac3eddfc66e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:36.538440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e34dc3-8bc2-4207-aeb7-bdcb09dc07b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:36.539141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e34dc3-8bc2-4207-aeb7-bdcb09dc07b1 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:36.545207 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e34dc3-8bc2-4207-aeb7-bdcb09dc07b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:36.547426 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e34dc3-8bc2-4207-aeb7-bdcb09dc07b1 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:36.548319 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e34dc3-8bc2-4207-aeb7-bdcb09dc07b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:36.553175 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e34dc3-8bc2-4207-aeb7-bdcb09dc07b1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:36.698321 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:36.701215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1264/2556] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:02:36 2022] POST /identity/v3/users => generated 293 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:02:36.705165 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea06a33-472c-43e8-86bf-34995affe1db admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:36.717524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea06a33-472c-43e8-86bf-34995affe1db admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:36.728843 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03478b-5bd5-4b9e-b1e6-aea5c1d14024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:36.730004 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03478b-5bd5-4b9e-b1e6-aea5c1d14024 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:36.730505 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03478b-5bd5-4b9e-b1e6-aea5c1d14024 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:36.731033 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03478b-5bd5-4b9e-b1e6-aea5c1d14024 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:36.731971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a03478b-5bd5-4b9e-b1e6-aea5c1d14024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:36.734102 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a03478b-5bd5-4b9e-b1e6-aea5c1d14024 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:36.748384 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1293/2557] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:36.753344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e34dc3-8bc2-4207-aeb7-bdcb09dc07b1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:36.775515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e34dc3-8bc2-4207-aeb7-bdcb09dc07b1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:36.795946 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640aa76-0ba1-43ab-9a47-3f573c3ada25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:36.795946 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640aa76-0ba1-43ab-9a47-3f573c3ada25 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:36.795946 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640aa76-0ba1-43ab-9a47-3f573c3ada25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:36.795946 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640aa76-0ba1-43ab-9a47-3f573c3ada25 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:36.796658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2640aa76-0ba1-43ab-9a47-3f573c3ada25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:36.796658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2640aa76-0ba1-43ab-9a47-3f573c3ada25 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:36.811467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1265/2558] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:36.813051 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03478b-5bd5-4b9e-b1e6-aea5c1d14024 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:36.835016 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03478b-5bd5-4b9e-b1e6-aea5c1d14024 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:36.858516 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d1d2c0-6ec8-4b9f-969b-3d810bbf6bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:36.860034 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1d2c0-6ec8-4b9f-969b-3d810bbf6bf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:36.862062 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1d2c0-6ec8-4b9f-969b-3d810bbf6bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:36.862561 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1d2c0-6ec8-4b9f-969b-3d810bbf6bf4 admin admin] PATH_INFO: `/v3/projects/0a79feb1dc7a4e499a4cf68b397b4c0b/users/0f46bf7ce65e47a793b3f1e7d33b0229/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:36.884935 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d1d2c0-6ec8-4b9f-969b-3d810bbf6bf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a79feb1dc7a4e499a4cf68b397b4c0b, user_id=0f46bf7ce65e47a793b3f1e7d33b0229, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:36.887924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d1d2c0-6ec8-4b9f-969b-3d810bbf6bf4 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:37.016814 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1294/2559] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:36 2022] PUT /identity/v3/projects/0a79feb1dc7a4e499a4cf68b397b4c0b/users/0f46bf7ce65e47a793b3f1e7d33b0229/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:37.028248 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ea770-27d1-47de-983e-3b6b49955ed3 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:37.029438 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ea770-27d1-47de-983e-3b6b49955ed3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:37.029843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ea770-27d1-47de-983e-3b6b49955ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:37.092016 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c95ea770-27d1-47de-983e-3b6b49955ed3 None None] MFA Rules not processed for user `0f46bf7ce65e47a793b3f1e7d33b0229`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:37.094348 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c95ea770-27d1-47de-983e-3b6b49955ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:37.194798 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1266/2560] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:37.195202 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d1d2c0-6ec8-4b9f-969b-3d810bbf6bf4 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:37.247687 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d1d2c0-6ec8-4b9f-969b-3d810bbf6bf4 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:37.258617 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb9bdc-1c6c-4405-b7ed-b8ee4960f0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:37.258617 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb9bdc-1c6c-4405-b7ed-b8ee4960f0a3 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:37.258617 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb9bdc-1c6c-4405-b7ed-b8ee4960f0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:37.258617 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb9bdc-1c6c-4405-b7ed-b8ee4960f0a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:37.259120 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb9bdc-1c6c-4405-b7ed-b8ee4960f0a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:37.259120 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb9bdc-1c6c-4405-b7ed-b8ee4960f0a3 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:37.790773 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1295/2561] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:02:37 2022] POST /identity/v3/projects => generated 385 bytes in 608 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:37.792881 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ea770-27d1-47de-983e-3b6b49955ed3 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:37.804968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ea770-27d1-47de-983e-3b6b49955ed3 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:37.920901 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d90133-9ca9-4053-8561-81e9b0fd69c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:37.921962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d90133-9ca9-4053-8561-81e9b0fd69c9 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:37.921962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d90133-9ca9-4053-8561-81e9b0fd69c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:37.922173 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d90133-9ca9-4053-8561-81e9b0fd69c9 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:37.922794 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d90133-9ca9-4053-8561-81e9b0fd69c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:37.926126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d90133-9ca9-4053-8561-81e9b0fd69c9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:38.052732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1267/2562] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:02:37 2022] POST /identity/v3/users => generated 299 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:38.057037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb9bdc-1c6c-4405-b7ed-b8ee4960f0a3 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:38.073646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb9bdc-1c6c-4405-b7ed-b8ee4960f0a3 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:38.094891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68317d90-23be-4ede-a4f7-a136002ff80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:38.095805 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68317d90-23be-4ede-a4f7-a136002ff80e admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:38.095805 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68317d90-23be-4ede-a4f7-a136002ff80e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:38.096182 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68317d90-23be-4ede-a4f7-a136002ff80e admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:38.096775 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68317d90-23be-4ede-a4f7-a136002ff80e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:38.098403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68317d90-23be-4ede-a4f7-a136002ff80e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:38.138700 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1296/2563] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:38.142584 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d90133-9ca9-4053-8561-81e9b0fd69c9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:38.155678 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d90133-9ca9-4053-8561-81e9b0fd69c9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:38.175283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784c4ec-3e2e-4e3e-9be9-743582e87cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:38.175806 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784c4ec-3e2e-4e3e-9be9-743582e87cf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:38.176301 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784c4ec-3e2e-4e3e-9be9-743582e87cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:38.176576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784c4ec-3e2e-4e3e-9be9-743582e87cf3 admin admin] PATH_INFO: `/v3/projects/5a3968ae03fa4fa9b660146809cf42ae/users/b8aa363d9dec46dfa2a4c4733db511da/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:38.207961 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784c4ec-3e2e-4e3e-9be9-743582e87cf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a3968ae03fa4fa9b660146809cf42ae, user_id=b8aa363d9dec46dfa2a4c4733db511da, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:38.211803 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784c4ec-3e2e-4e3e-9be9-743582e87cf3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:38.333812 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1268/2564] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:38 2022] PUT /identity/v3/projects/5a3968ae03fa4fa9b660146809cf42ae/users/b8aa363d9dec46dfa2a4c4733db511da/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:38.337870 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68317d90-23be-4ede-a4f7-a136002ff80e admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:38.350283 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68317d90-23be-4ede-a4f7-a136002ff80e admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:38.399017 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:38.435482 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aadd31-8dba-4770-aa8f-178925838ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:38.437343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aadd31-8dba-4770-aa8f-178925838ddc admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:38.437792 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aadd31-8dba-4770-aa8f-178925838ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:38.438260 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aadd31-8dba-4770-aa8f-178925838ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:38.439089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aadd31-8dba-4770-aa8f-178925838ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:38.441969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aadd31-8dba-4770-aa8f-178925838ddc admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:38.456070 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1297/2565] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:02:38.460167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784c4ec-3e2e-4e3e-9be9-743582e87cf3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:38.473224 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784c4ec-3e2e-4e3e-9be9-743582e87cf3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:38.483774 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7bdff-6447-4804-91d5-cb4c62fdebff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:38.485000 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7bdff-6447-4804-91d5-cb4c62fdebff admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:38.486157 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7bdff-6447-4804-91d5-cb4c62fdebff admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:38.487513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7bdff-6447-4804-91d5-cb4c62fdebff admin admin] PATH_INFO: `/v3/projects/5a3968ae03fa4fa9b660146809cf42ae/users/b8aa363d9dec46dfa2a4c4733db511da/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:38.494448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7bdff-6447-4804-91d5-cb4c62fdebff admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a3968ae03fa4fa9b660146809cf42ae, user_id=b8aa363d9dec46dfa2a4c4733db511da, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:38.497505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7bdff-6447-4804-91d5-cb4c62fdebff admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:38.525107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1269/2566] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:38 2022] PUT /identity/v3/projects/5a3968ae03fa4fa9b660146809cf42ae/users/b8aa363d9dec46dfa2a4c4733db511da/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:02:38.526035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12af81-fb70-4ab4-9284-3b3f04754d35 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:38.527522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12af81-fb70-4ab4-9284-3b3f04754d35 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:38.527966 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12af81-fb70-4ab4-9284-3b3f04754d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:38.586035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-8c12af81-fb70-4ab4-9284-3b3f04754d35 None None] MFA Rules not processed for user `b8aa363d9dec46dfa2a4c4733db511da`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:38.587665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8c12af81-fb70-4ab4-9284-3b3f04754d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:38.678054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7bdff-6447-4804-91d5-cb4c62fdebff admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:38.683579 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ecc7bdff-6447-4804-91d5-cb4c62fdebff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:38.763744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1298/2567] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:02:38.794726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12af81-fb70-4ab4-9284-3b3f04754d35 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:38.797054 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8c12af81-fb70-4ab4-9284-3b3f04754d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:38.833555 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:38.919542 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7bdff-6447-4804-91d5-cb4c62fdebff admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:38.935358 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb959952-354e-433f-8ef7-57bcc63772bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:38.937468 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb959952-354e-433f-8ef7-57bcc63772bc service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:38.937775 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb959952-354e-433f-8ef7-57bcc63772bc service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:38.938054 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb959952-354e-433f-8ef7-57bcc63772bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:38.944953 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fb959952-354e-433f-8ef7-57bcc63772bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:38.972879 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb959952-354e-433f-8ef7-57bcc63772bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:38.976608 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12af81-fb70-4ab4-9284-3b3f04754d35 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:38.980240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb959952-354e-433f-8ef7-57bcc63772bc service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:38.989277 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154aaf2-6bf3-4732-9f5a-69b0504d8908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:38.989904 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154aaf2-6bf3-4732-9f5a-69b0504d8908 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:38.990182 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154aaf2-6bf3-4732-9f5a-69b0504d8908 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:38.990453 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154aaf2-6bf3-4732-9f5a-69b0504d8908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:39.015082 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1270/2568] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:39.033748 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154aaf2-6bf3-4732-9f5a-69b0504d8908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:39.035808 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154aaf2-6bf3-4732-9f5a-69b0504d8908 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:39.105457 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1299/2569] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:40.769656 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb959952-354e-433f-8ef7-57bcc63772bc service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:40.844526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb959952-354e-433f-8ef7-57bcc63772bc service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:40.850192 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41cb45f-0b8c-4cd1-9ed2-8c31dba23927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:40.850678 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cb45f-0b8c-4cd1-9ed2-8c31dba23927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:40.851043 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cb45f-0b8c-4cd1-9ed2-8c31dba23927 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:40.851392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cb45f-0b8c-4cd1-9ed2-8c31dba23927 admin admin] PATH_INFO: `/v3/users/0f46bf7ce65e47a793b3f1e7d33b0229` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:40.853715 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41cb45f-0b8c-4cd1-9ed2-8c31dba23927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f46bf7ce65e47a793b3f1e7d33b0229)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:40.855132 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41cb45f-0b8c-4cd1-9ed2-8c31dba23927 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:41.048428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f41cb45f-0b8c-4cd1-9ed2-8c31dba23927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f46bf7ce65e47a793b3f1e7d33b0229'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:41.057115 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f41cb45f-0b8c-4cd1-9ed2-8c31dba23927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f46bf7ce65e47a793b3f1e7d33b0229'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:41.057920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f41cb45f-0b8c-4cd1-9ed2-8c31dba23927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f46bf7ce65e47a793b3f1e7d33b0229'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:41.178916 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f41cb45f-0b8c-4cd1-9ed2-8c31dba23927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f46bf7ce65e47a793b3f1e7d33b0229'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:41.213868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-f41cb45f-0b8c-4cd1-9ed2-8c31dba23927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f46bf7ce65e47a793b3f1e7d33b0229'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:41.226627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1271/2570] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:02:40 2022] DELETE /identity/v3/users/0f46bf7ce65e47a793b3f1e7d33b0229 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:02:41.232192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bdacc-94f1-4c74-acb0-1cfd9e88628a None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:41.232645 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bdacc-94f1-4c74-acb0-1cfd9e88628a None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:41.232825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bdacc-94f1-4c74-acb0-1cfd9e88628a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:41.268969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-cc1bdacc-94f1-4c74-acb0-1cfd9e88628a None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:41.270352 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-cc1bdacc-94f1-4c74-acb0-1cfd9e88628a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:41.348256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:41.357562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1300/2571] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:41.372300 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41cb45f-0b8c-4cd1-9ed2-8c31dba23927 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:41.376548 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f41cb45f-0b8c-4cd1-9ed2-8c31dba23927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:41.438625 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41cb45f-0b8c-4cd1-9ed2-8c31dba23927 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:41.442543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01932be2-7de2-4a5f-90ed-833485df86bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:41.444410 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01932be2-7de2-4a5f-90ed-833485df86bc service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:41.444410 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01932be2-7de2-4a5f-90ed-833485df86bc service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:41.444410 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01932be2-7de2-4a5f-90ed-833485df86bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:41.453027 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:41.455716 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01932be2-7de2-4a5f-90ed-833485df86bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:41.457645 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01932be2-7de2-4a5f-90ed-833485df86bc service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:41.469421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1272/2572] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:42.080185 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bdacc-94f1-4c74-acb0-1cfd9e88628a None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:42.080637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-cc1bdacc-94f1-4c74-acb0-1cfd9e88628a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:42.100963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bdacc-94f1-4c74-acb0-1cfd9e88628a None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:42.105078 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e27b66-6c76-4982-8c63-5addc526b2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:42.106178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e27b66-6c76-4982-8c63-5addc526b2c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:42.106458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e27b66-6c76-4982-8c63-5addc526b2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:42.106782 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e27b66-6c76-4982-8c63-5addc526b2c8 admin admin] PATH_INFO: `/v3/projects/0a79feb1dc7a4e499a4cf68b397b4c0b` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:42.108200 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e27b66-6c76-4982-8c63-5addc526b2c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a79feb1dc7a4e499a4cf68b397b4c0b)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:42.110146 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e27b66-6c76-4982-8c63-5addc526b2c8 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:42.226038 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a3e27b66-6c76-4982-8c63-5addc526b2c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a79feb1dc7a4e499a4cf68b397b4c0b', 'request_id': 'req-a3e27b66-6c76-4982-8c63-5addc526b2c8'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:42.231945 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a3e27b66-6c76-4982-8c63-5addc526b2c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a79feb1dc7a4e499a4cf68b397b4c0b', 'request_id': 'req-a3e27b66-6c76-4982-8c63-5addc526b2c8'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:42.338282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a3e27b66-6c76-4982-8c63-5addc526b2c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a79feb1dc7a4e499a4cf68b397b4c0b', 'request_id': 'req-a3e27b66-6c76-4982-8c63-5addc526b2c8'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:42.352674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a3e27b66-6c76-4982-8c63-5addc526b2c8 admin admin] The token cache is being invalidate because project 0a79feb1dc7a4e499a4cf68b397b4c0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:02:42.353100 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a3e27b66-6c76-4982-8c63-5addc526b2c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:42.353770 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a3e27b66-6c76-4982-8c63-5addc526b2c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:42.355805 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1301/2573] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:02:42 2022] DELETE /identity/v3/projects/0a79feb1dc7a4e499a4cf68b397b4c0b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:42.360523 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01932be2-7de2-4a5f-90ed-833485df86bc service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:42.370376 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01932be2-7de2-4a5f-90ed-833485df86bc service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:42.435265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55999805-a11d-49f5-a5bd-5dca784d76c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:42.436205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55999805-a11d-49f5-a5bd-5dca784d76c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:42.436738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55999805-a11d-49f5-a5bd-5dca784d76c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:42.437145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55999805-a11d-49f5-a5bd-5dca784d76c0 admin admin] PATH_INFO: `/v3/users/b8aa363d9dec46dfa2a4c4733db511da` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:42.438469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55999805-a11d-49f5-a5bd-5dca784d76c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8aa363d9dec46dfa2a4c4733db511da)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:42.440669 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55999805-a11d-49f5-a5bd-5dca784d76c0 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:42.696287 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-55999805-a11d-49f5-a5bd-5dca784d76c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8aa363d9dec46dfa2a4c4733db511da'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:42.703957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-55999805-a11d-49f5-a5bd-5dca784d76c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8aa363d9dec46dfa2a4c4733db511da'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:42.704775 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-55999805-a11d-49f5-a5bd-5dca784d76c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8aa363d9dec46dfa2a4c4733db511da'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:42.823130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-55999805-a11d-49f5-a5bd-5dca784d76c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8aa363d9dec46dfa2a4c4733db511da'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:42.842454 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-55999805-a11d-49f5-a5bd-5dca784d76c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8aa363d9dec46dfa2a4c4733db511da'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:42.854711 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1273/2574] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:02:42 2022] DELETE /identity/v3/users/b8aa363d9dec46dfa2a4c4733db511da => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:43.593664 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e27b66-6c76-4982-8c63-5addc526b2c8 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:43.594659 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a3e27b66-6c76-4982-8c63-5addc526b2c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:43.611501 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e27b66-6c76-4982-8c63-5addc526b2c8 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:43.683465 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea955bec-30db-4a02-ab4c-6ffb2fdaebb2 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:43.684330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea955bec-30db-4a02-ab4c-6ffb2fdaebb2 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:43.684786 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea955bec-30db-4a02-ab4c-6ffb2fdaebb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:43.715465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee5453-a2ed-489b-b2c3-c2798f1c6626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:43.716705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee5453-a2ed-489b-b2c3-c2798f1c6626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:43.717487 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee5453-a2ed-489b-b2c3-c2798f1c6626 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:43.718143 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee5453-a2ed-489b-b2c3-c2798f1c6626 admin admin] PATH_INFO: `/v3/projects/5a3968ae03fa4fa9b660146809cf42ae` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:43.719772 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee5453-a2ed-489b-b2c3-c2798f1c6626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a3968ae03fa4fa9b660146809cf42ae)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:43.723592 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee5453-a2ed-489b-b2c3-c2798f1c6626 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:43.729363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ea955bec-30db-4a02-ab4c-6ffb2fdaebb2 None None] MFA Rules not processed for user `643bb62a05e64c6c844fda8ec2ee7e41`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:43.731033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ea955bec-30db-4a02-ab4c-6ffb2fdaebb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:43.812003 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:43.818342 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1274/2575] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:43.831741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea955bec-30db-4a02-ab4c-6ffb2fdaebb2 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:43.834113 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ea955bec-30db-4a02-ab4c-6ffb2fdaebb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:43.881056 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-18ee5453-a2ed-489b-b2c3-c2798f1c6626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a3968ae03fa4fa9b660146809cf42ae', 'request_id': 'req-18ee5453-a2ed-489b-b2c3-c2798f1c6626'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:43.889912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-18ee5453-a2ed-489b-b2c3-c2798f1c6626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a3968ae03fa4fa9b660146809cf42ae', 'request_id': 'req-18ee5453-a2ed-489b-b2c3-c2798f1c6626'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:43.918653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea955bec-30db-4a02-ab4c-6ffb2fdaebb2 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:43.966304 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-18ee5453-a2ed-489b-b2c3-c2798f1c6626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a3968ae03fa4fa9b660146809cf42ae', 'request_id': 'req-18ee5453-a2ed-489b-b2c3-c2798f1c6626'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:43.981041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-18ee5453-a2ed-489b-b2c3-c2798f1c6626 admin admin] The token cache is being invalidate because project 5a3968ae03fa4fa9b660146809cf42ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:02:43.981595 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-18ee5453-a2ed-489b-b2c3-c2798f1c6626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:43.982890 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e99fa-9df1-4da6-b8d2-1cff0dfebbd9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfda7307e703427faaf599e32b91bc91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:43.983343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-18ee5453-a2ed-489b-b2c3-c2798f1c6626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:02:43.983554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e99fa-9df1-4da6-b8d2-1cff0dfebbd9 service glance] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:43.984024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e99fa-9df1-4da6-b8d2-1cff0dfebbd9 service glance] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:43.984477 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e99fa-9df1-4da6-b8d2-1cff0dfebbd9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:43.986409 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1302/2576] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:02:43 2022] DELETE /identity/v3/projects/5a3968ae03fa4fa9b660146809cf42ae => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:43.988748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-779e99fa-9df1-4da6-b8d2-1cff0dfebbd9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:43.993261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70719e1f-8679-4af0-b077-52a6cbd744f3 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:43.993879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70719e1f-8679-4af0-b077-52a6cbd744f3 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:43.994155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70719e1f-8679-4af0-b077-52a6cbd744f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:44.015951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:44.023323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-70719e1f-8679-4af0-b077-52a6cbd744f3 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:44.024756 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-70719e1f-8679-4af0-b077-52a6cbd744f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:44.085480 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e99fa-9df1-4da6-b8d2-1cff0dfebbd9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:44.087305 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e99fa-9df1-4da6-b8d2-1cff0dfebbd9 service glance] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:44.099396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1275/2577] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:02:44.103047 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1303/2578] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:44.111740 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58dcb2a-a735-476b-92e4-674238ce85b9 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:44.112154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58dcb2a-a735-476b-92e4-674238ce85b9 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:44.112427 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58dcb2a-a735-476b-92e4-674238ce85b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:44.133887 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b58dcb2a-a735-476b-92e4-674238ce85b9 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:44.135125 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b58dcb2a-a735-476b-92e4-674238ce85b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:44.143869 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1276/2579] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:44.148230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70719e1f-8679-4af0-b077-52a6cbd744f3 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:44.158140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70719e1f-8679-4af0-b077-52a6cbd744f3 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:44.162862 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f214c-7471-49b3-8685-e80fe966ac6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:44.163496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f214c-7471-49b3-8685-e80fe966ac6c admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:44.163782 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f214c-7471-49b3-8685-e80fe966ac6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:44.164069 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f214c-7471-49b3-8685-e80fe966ac6c admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:44.164770 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f214c-7471-49b3-8685-e80fe966ac6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:02:44.165074 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f214c-7471-49b3-8685-e80fe966ac6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:44.166792 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f214c-7471-49b3-8685-e80fe966ac6c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:44.178640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1304/2580] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:02:44.183672 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58dcb2a-a735-476b-92e4-674238ce85b9 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:44.192863 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58dcb2a-a735-476b-92e4-674238ce85b9 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:44.198156 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf02d0a-c229-4f1c-9e1f-388c77e342bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:44.198835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf02d0a-c229-4f1c-9e1f-388c77e342bc admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:44.199135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf02d0a-c229-4f1c-9e1f-388c77e342bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:44.199475 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf02d0a-c229-4f1c-9e1f-388c77e342bc admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:44.200126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf02d0a-c229-4f1c-9e1f-388c77e342bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:44.202044 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf02d0a-c229-4f1c-9e1f-388c77e342bc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:44.324568 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1277/2581] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:02:44 2022] POST /identity/v3/projects => generated 363 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:44.325779 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f214c-7471-49b3-8685-e80fe966ac6c admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:44.342787 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f214c-7471-49b3-8685-e80fe966ac6c admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:44.407980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546757a-f906-45a9-afcd-4416eec4516d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:44.408574 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546757a-f906-45a9-afcd-4416eec4516d admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:44.408850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546757a-f906-45a9-afcd-4416eec4516d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:44.409155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546757a-f906-45a9-afcd-4416eec4516d admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:44.409780 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5546757a-f906-45a9-afcd-4416eec4516d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:44.411714 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5546757a-f906-45a9-afcd-4416eec4516d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:44.449962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fba52-06ce-437d-b2b5-1c15ab917967 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:44.450255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fba52-06ce-437d-b2b5-1c15ab917967 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:44.450532 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fba52-06ce-437d-b2b5-1c15ab917967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:44.481002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e99fba52-06ce-437d-b2b5-1c15ab917967 None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:44.483968 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e99fba52-06ce-437d-b2b5-1c15ab917967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:44.513338 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1305/2582] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:02:44 2022] POST /identity/v3/users => generated 282 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:02:44.517697 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546757a-f906-45a9-afcd-4416eec4516d admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:44.529178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546757a-f906-45a9-afcd-4416eec4516d admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:44.536299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5992d4-0665-4bce-a9cd-1892da52fbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:44.536876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5992d4-0665-4bce-a9cd-1892da52fbd9 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:44.537162 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5992d4-0665-4bce-a9cd-1892da52fbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:44.537357 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:44.537489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5992d4-0665-4bce-a9cd-1892da52fbd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:44.538076 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5992d4-0665-4bce-a9cd-1892da52fbd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:44.540346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5992d4-0665-4bce-a9cd-1892da52fbd9 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:44.552948 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1306/2583] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:02:44.557265 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5992d4-0665-4bce-a9cd-1892da52fbd9 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:44.568358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5992d4-0665-4bce-a9cd-1892da52fbd9 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:44.574250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33437680-de2e-4b7b-94c0-b61c629e3ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:44.575037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33437680-de2e-4b7b-94c0-b61c629e3ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:44.575653 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33437680-de2e-4b7b-94c0-b61c629e3ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:44.576333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33437680-de2e-4b7b-94c0-b61c629e3ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:44.577294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33437680-de2e-4b7b-94c0-b61c629e3ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:44.579315 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33437680-de2e-4b7b-94c0-b61c629e3ca0 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:44.579846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1278/2584] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 30 15:02:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:44.593427 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1307/2585] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:02:44.600050 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fba52-06ce-437d-b2b5-1c15ab917967 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:44.602974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33437680-de2e-4b7b-94c0-b61c629e3ca0 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:44.603444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e99fba52-06ce-437d-b2b5-1c15ab917967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:44.624767 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33437680-de2e-4b7b-94c0-b61c629e3ca0 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:44.631725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685a49b-4feb-4d02-b430-61475925149c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:44.632510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685a49b-4feb-4d02-b430-61475925149c admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:44.632797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685a49b-4feb-4d02-b430-61475925149c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:44.634032 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685a49b-4feb-4d02-b430-61475925149c admin admin] PATH_INFO: `/v3/projects/1279187a4abc458fb2d4fc14375504cf/users/46ed5dde54a74913b501373fb57c25cb/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:44.660938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6685a49b-4feb-4d02-b430-61475925149c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1279187a4abc458fb2d4fc14375504cf, user_id=46ed5dde54a74913b501373fb57c25cb, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:44.664630 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6685a49b-4feb-4d02-b430-61475925149c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:44.706428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fba52-06ce-437d-b2b5-1c15ab917967 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:44.719858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff06cbe-b935-4bbf-ae06-84745975f113 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:44.720491 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff06cbe-b935-4bbf-ae06-84745975f113 service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:44.720768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff06cbe-b935-4bbf-ae06-84745975f113 service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:44.721045 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff06cbe-b935-4bbf-ae06-84745975f113 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:44.733145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff06cbe-b935-4bbf-ae06-84745975f113 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:44.735256 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff06cbe-b935-4bbf-ae06-84745975f113 service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:44.745686 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1279/2586] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 15:02:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:02:44.745892 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1308/2587] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:02:44 2022] PUT /identity/v3/projects/1279187a4abc458fb2d4fc14375504cf/users/46ed5dde54a74913b501373fb57c25cb/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:02:44.815038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e5d16-43ba-496a-a5bc-02367065b19b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:44.815383 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e5d16-43ba-496a-a5bc-02367065b19b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:44.815662 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e5d16-43ba-496a-a5bc-02367065b19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:44.845123 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-535e5d16-43ba-496a-a5bc-02367065b19b None None] MFA Rules not processed for user `46ed5dde54a74913b501373fb57c25cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:44.846321 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-535e5d16-43ba-496a-a5bc-02367065b19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:44.919621 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1280/2588] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:44.931868 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685a49b-4feb-4d02-b430-61475925149c admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:44.932347 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6685a49b-4feb-4d02-b430-61475925149c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:45.005261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:45.029326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685a49b-4feb-4d02-b430-61475925149c admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:45.033652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bd140-f455-4954-afe3-9ce83ccda2a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:45.034396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bd140-f455-4954-afe3-9ce83ccda2a9 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:45.034775 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bd140-f455-4954-afe3-9ce83ccda2a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:45.035179 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bd140-f455-4954-afe3-9ce83ccda2a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:45.045822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bd140-f455-4954-afe3-9ce83ccda2a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:45.047808 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bd140-f455-4954-afe3-9ce83ccda2a9 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:45.057939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1309/2589] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:47.994056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e5d16-43ba-496a-a5bc-02367065b19b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:47.997225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-535e5d16-43ba-496a-a5bc-02367065b19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:48.166837 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e5d16-43ba-496a-a5bc-02367065b19b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:48.175465 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0466e-cec0-4a9e-b017-d8648cad9a26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4f11655cd51438e9652869f74ffa975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:48.176342 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0466e-cec0-4a9e-b017-d8648cad9a26 service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:48.176684 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0466e-cec0-4a9e-b017-d8648cad9a26 service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:48.177044 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0466e-cec0-4a9e-b017-d8648cad9a26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:48.181537 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2cb0466e-cec0-4a9e-b017-d8648cad9a26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:48.200768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:48.207379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb0466e-cec0-4a9e-b017-d8648cad9a26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:48.209006 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb0466e-cec0-4a9e-b017-d8648cad9a26 service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:48.222608 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1281/2590] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:53.465732 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707c07a-2536-4ef9-b23c-83dfc7fbfc2d None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:53.466244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707c07a-2536-4ef9-b23c-83dfc7fbfc2d None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:53.466342 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707c07a-2536-4ef9-b23c-83dfc7fbfc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:53.519387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-5707c07a-2536-4ef9-b23c-83dfc7fbfc2d None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:53.521067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-5707c07a-2536-4ef9-b23c-83dfc7fbfc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:53.610727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1310/2591] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 30 15:02:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:53.681912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0466e-cec0-4a9e-b017-d8648cad9a26 service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:53.807781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0466e-cec0-4a9e-b017-d8648cad9a26 service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:53.815841 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44955c53-977b-4a80-ad17-99fbf180e6b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:53.816323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44955c53-977b-4a80-ad17-99fbf180e6b5 service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:53.816605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44955c53-977b-4a80-ad17-99fbf180e6b5 service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:53.816878 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44955c53-977b-4a80-ad17-99fbf180e6b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:53.833667 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44955c53-977b-4a80-ad17-99fbf180e6b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:53.836983 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44955c53-977b-4a80-ad17-99fbf180e6b5 service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:53.851417 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1282/2592] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 15:02:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:02:54.050468 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707c07a-2536-4ef9-b23c-83dfc7fbfc2d None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:54.052054 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-5707c07a-2536-4ef9-b23c-83dfc7fbfc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:54.174105 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707c07a-2536-4ef9-b23c-83dfc7fbfc2d None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:54.197090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c671e-4593-4bb8-a0d0-0cced1382baf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:54.197678 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c671e-4593-4bb8-a0d0-0cced1382baf service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:54.197951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c671e-4593-4bb8-a0d0-0cced1382baf service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:54.198218 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c671e-4593-4bb8-a0d0-0cced1382baf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:54.200095 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-dd7c671e-4593-4bb8-a0d0-0cced1382baf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:54.235720 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c671e-4593-4bb8-a0d0-0cced1382baf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:54.235720 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c671e-4593-4bb8-a0d0-0cced1382baf service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:54.263159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1311/2593] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:58.017037 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42a3cf-ceb4-4626-afab-7145144e573f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:58.017644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42a3cf-ceb4-4626-afab-7145144e573f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:58.017644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42a3cf-ceb4-4626-afab-7145144e573f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:58.038956 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ff42a3cf-ceb4-4626-afab-7145144e573f None None] MFA Rules not processed for user `2bc1bd791342420c99321b3ebd73bd57`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:02:58.040191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ff42a3cf-ceb4-4626-afab-7145144e573f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:58.106797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1283/2594] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:02:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:02:58.368850 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c671e-4593-4bb8-a0d0-0cced1382baf service nova] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:58.370368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-dd7c671e-4593-4bb8-a0d0-0cced1382baf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:58.426196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c671e-4593-4bb8-a0d0-0cced1382baf service nova] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:58.429924 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e7ad3-2000-4a8c-89c2-8a6b9de654df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:58.431012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e7ad3-2000-4a8c-89c2-8a6b9de654df service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:58.431310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e7ad3-2000-4a8c-89c2-8a6b9de654df service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:58.431574 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e7ad3-2000-4a8c-89c2-8a6b9de654df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:58.433386 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1d5e7ad3-2000-4a8c-89c2-8a6b9de654df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:58.442704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e7ad3-2000-4a8c-89c2-8a6b9de654df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:58.444145 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e7ad3-2000-4a8c-89c2-8a6b9de654df service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:58.452649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1312/2595] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:03:04.710458 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42a3cf-ceb4-4626-afab-7145144e573f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:04.712288 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ff42a3cf-ceb4-4626-afab-7145144e573f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:04.726027 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42a3cf-ceb4-4626-afab-7145144e573f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:04.730315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a0c5d-9469-4c74-859a-80fc790e9089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:04.732783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a0c5d-9469-4c74-859a-80fc790e9089 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:04.732783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a0c5d-9469-4c74-859a-80fc790e9089 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:04.732783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a0c5d-9469-4c74-859a-80fc790e9089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:04.733334 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-357a0c5d-9469-4c74-859a-80fc790e9089 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:04.803745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a0c5d-9469-4c74-859a-80fc790e9089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:04.805955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a0c5d-9469-4c74-859a-80fc790e9089 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:04.819911 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1284/2596] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:03:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:03:08.449132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=94 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:03:09.690434 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e7ad3-2000-4a8c-89c2-8a6b9de654df service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:09.692023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1d5e7ad3-2000-4a8c-89c2-8a6b9de654df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:09.709081 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e7ad3-2000-4a8c-89c2-8a6b9de654df service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:09.713775 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f6687-652f-47bc-b788-0491cea3af96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:09.714462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f6687-652f-47bc-b788-0491cea3af96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:09.714740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f6687-652f-47bc-b788-0491cea3af96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:09.715035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f6687-652f-47bc-b788-0491cea3af96 admin admin] PATH_INFO: `/v3/users/3bd31b8807fc450f8f134417cd535542` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:09.716277 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f6687-652f-47bc-b788-0491cea3af96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bd31b8807fc450f8f134417cd535542)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:09.718306 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f6687-652f-47bc-b788-0491cea3af96 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:09.968084 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b97f6687-652f-47bc-b788-0491cea3af96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bd31b8807fc450f8f134417cd535542'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:09.977105 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b97f6687-652f-47bc-b788-0491cea3af96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bd31b8807fc450f8f134417cd535542'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:09.996803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b97f6687-652f-47bc-b788-0491cea3af96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bd31b8807fc450f8f134417cd535542'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:09.997582 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b97f6687-652f-47bc-b788-0491cea3af96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bd31b8807fc450f8f134417cd535542'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:09.998343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b97f6687-652f-47bc-b788-0491cea3af96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bd31b8807fc450f8f134417cd535542'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:10.094540 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1313/2597] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:03:09 2022] DELETE /identity/v3/users/3bd31b8807fc450f8f134417cd535542 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:03:10.359251 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35e111-630d-4abb-b1fe-75be8b721560 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:10.360093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35e111-630d-4abb-b1fe-75be8b721560 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:10.360413 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35e111-630d-4abb-b1fe-75be8b721560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:10.403664 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-5b35e111-630d-4abb-b1fe-75be8b721560 None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:03:10.404935 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5b35e111-630d-4abb-b1fe-75be8b721560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:10.458849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:03:10.548898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1285/2598] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 30 15:03:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:03:10.569030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f6687-652f-47bc-b788-0491cea3af96 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:10.570522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b97f6687-652f-47bc-b788-0491cea3af96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:10.669355 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f6687-652f-47bc-b788-0491cea3af96 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:10.674835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4374cf-215b-4fb2-8f0f-23048a2528de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:10.675622 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4374cf-215b-4fb2-8f0f-23048a2528de service swift] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:10.676005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4374cf-215b-4fb2-8f0f-23048a2528de service swift] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:10.676590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4374cf-215b-4fb2-8f0f-23048a2528de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:10.687783 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4374cf-215b-4fb2-8f0f-23048a2528de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:10.689888 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4374cf-215b-4fb2-8f0f-23048a2528de service swift] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:10.707464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1314/2599] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 15:03:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:03:10.771022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35e111-630d-4abb-b1fe-75be8b721560 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:10.771022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-5b35e111-630d-4abb-b1fe-75be8b721560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:10.979289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35e111-630d-4abb-b1fe-75be8b721560 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:11.005721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b6abb-d951-4fec-9612-c8550d652d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:11.006744 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b6abb-d951-4fec-9612-c8550d652d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:11.007257 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b6abb-d951-4fec-9612-c8550d652d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:11.007767 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b6abb-d951-4fec-9612-c8550d652d26 admin admin] PATH_INFO: `/v3/projects/507c97f5d4a74853a3c50b4aef9e891c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:11.013244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b6abb-d951-4fec-9612-c8550d652d26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=507c97f5d4a74853a3c50b4aef9e891c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:11.016022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b6abb-d951-4fec-9612-c8550d652d26 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:11.208565 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9e0b6abb-d951-4fec-9612-c8550d652d26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '507c97f5d4a74853a3c50b4aef9e891c', 'request_id': 'req-9e0b6abb-d951-4fec-9612-c8550d652d26'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:11.218993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9e0b6abb-d951-4fec-9612-c8550d652d26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '507c97f5d4a74853a3c50b4aef9e891c', 'request_id': 'req-9e0b6abb-d951-4fec-9612-c8550d652d26'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:11.239283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9e0b6abb-d951-4fec-9612-c8550d652d26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '507c97f5d4a74853a3c50b4aef9e891c', 'request_id': 'req-9e0b6abb-d951-4fec-9612-c8550d652d26'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:11.392839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9e0b6abb-d951-4fec-9612-c8550d652d26 admin admin] The token cache is being invalidate because project 507c97f5d4a74853a3c50b4aef9e891c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:03:11.393577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9e0b6abb-d951-4fec-9612-c8550d652d26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:11.395619 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-9e0b6abb-d951-4fec-9612-c8550d652d26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:11.398282 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1286/2600] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:03:10 2022] DELETE /identity/v3/projects/507c97f5d4a74853a3c50b4aef9e891c => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:03:11.405957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee99f16-1a06-4476-9cc5-1234b133d307 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:11.411217 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee99f16-1a06-4476-9cc5-1234b133d307 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:11.411217 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee99f16-1a06-4476-9cc5-1234b133d307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:11.470950 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-7ee99f16-1a06-4476-9cc5-1234b133d307 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:03:11.472257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7ee99f16-1a06-4476-9cc5-1234b133d307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:11.711772 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1315/2601] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:03:11.720706 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9eb57-6b9e-4eb3-a5e2-65235a5ac303 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:11.724107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9eb57-6b9e-4eb3-a5e2-65235a5ac303 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:11.724693 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9eb57-6b9e-4eb3-a5e2-65235a5ac303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:11.770373 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-c6e9eb57-6b9e-4eb3-a5e2-65235a5ac303 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:03:11.774999 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c6e9eb57-6b9e-4eb3-a5e2-65235a5ac303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:11.803901 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1287/2602] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:03:11.813088 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee99f16-1a06-4476-9cc5-1234b133d307 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:11.818571 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:03:11.825122 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee99f16-1a06-4476-9cc5-1234b133d307 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:11.860344 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7c442-f003-4406-a696-b896d71c8563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:11.861097 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7c442-f003-4406-a696-b896d71c8563 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:11.861484 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7c442-f003-4406-a696-b896d71c8563 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:11.861847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7c442-f003-4406-a696-b896d71c8563 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:11.862754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7c442-f003-4406-a696-b896d71c8563 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:03:11.868181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7c442-f003-4406-a696-b896d71c8563 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:11.871742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7c442-f003-4406-a696-b896d71c8563 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:11.910705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1316/2603] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:03:11.915939 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e9eb57-6b9e-4eb3-a5e2-65235a5ac303 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:11.932992 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e9eb57-6b9e-4eb3-a5e2-65235a5ac303 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:11.944475 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40eef8-9c58-4687-bcad-3f99a0df3344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:11.946034 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40eef8-9c58-4687-bcad-3f99a0df3344 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:11.946898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40eef8-9c58-4687-bcad-3f99a0df3344 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:11.947914 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40eef8-9c58-4687-bcad-3f99a0df3344 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:11.948718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40eef8-9c58-4687-bcad-3f99a0df3344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:11.964046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d40eef8-9c58-4687-bcad-3f99a0df3344 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:12.120321 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1288/2604] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:03:11 2022] POST /identity/v3/projects => generated 385 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:03:12.123871 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7c442-f003-4406-a696-b896d71c8563 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:12.138822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7c442-f003-4406-a696-b896d71c8563 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:12.253281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d4ac3-fcca-4fef-9218-1256fff79d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:12.253990 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d4ac3-fcca-4fef-9218-1256fff79d6d admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:12.254337 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d4ac3-fcca-4fef-9218-1256fff79d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:12.254662 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d4ac3-fcca-4fef-9218-1256fff79d6d admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:12.255349 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d4ac3-fcca-4fef-9218-1256fff79d6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:12.257303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d4ac3-fcca-4fef-9218-1256fff79d6d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:12.402226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1317/2605] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:03:12 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:03:12.402408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40eef8-9c58-4687-bcad-3f99a0df3344 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:12.418225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40eef8-9c58-4687-bcad-3f99a0df3344 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:12.425863 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b82297-787a-4624-b65a-5147b94e1a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:12.427014 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b82297-787a-4624-b65a-5147b94e1a4e admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:12.427316 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b82297-787a-4624-b65a-5147b94e1a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:12.427825 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b82297-787a-4624-b65a-5147b94e1a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:12.428965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b82297-787a-4624-b65a-5147b94e1a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:12.431027 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b82297-787a-4624-b65a-5147b94e1a4e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:12.444307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1289/2606] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:03:12.448350 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d4ac3-fcca-4fef-9218-1256fff79d6d admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:12.461169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d4ac3-fcca-4fef-9218-1256fff79d6d admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:12.467051 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6be9a0-80c4-4231-b08f-210c8e80eec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:12.468259 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6be9a0-80c4-4231-b08f-210c8e80eec2 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:12.469098 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6be9a0-80c4-4231-b08f-210c8e80eec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:12.469559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6be9a0-80c4-4231-b08f-210c8e80eec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:12.470531 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6be9a0-80c4-4231-b08f-210c8e80eec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:12.472416 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6be9a0-80c4-4231-b08f-210c8e80eec2 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:12.487650 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1318/2607] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:03:12.491955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b82297-787a-4624-b65a-5147b94e1a4e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:12.504733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b82297-787a-4624-b65a-5147b94e1a4e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:12.511186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07faffe-4715-4f7e-a6a9-2664a1f58652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:12.512198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07faffe-4715-4f7e-a6a9-2664a1f58652 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:12.512690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07faffe-4715-4f7e-a6a9-2664a1f58652 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:12.513247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07faffe-4715-4f7e-a6a9-2664a1f58652 admin admin] PATH_INFO: `/v3/projects/74d147248b7c46bb821f0ca4cd7efd0c/users/6a9784d49f8d45e38f096de3b2401d36/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:12.533720 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07faffe-4715-4f7e-a6a9-2664a1f58652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d147248b7c46bb821f0ca4cd7efd0c, user_id=6a9784d49f8d45e38f096de3b2401d36, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:12.538090 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07faffe-4715-4f7e-a6a9-2664a1f58652 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:12.544341 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:03:12.629332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1290/2608] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:03:12 2022] PUT /identity/v3/projects/74d147248b7c46bb821f0ca4cd7efd0c/users/6a9784d49f8d45e38f096de3b2401d36/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:03:12.634545 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1304b6-0d18-4066-b216-2e0847a7d396 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:12.636842 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1304b6-0d18-4066-b216-2e0847a7d396 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:12.639553 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1304b6-0d18-4066-b216-2e0847a7d396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:12.687687 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-bd1304b6-0d18-4066-b216-2e0847a7d396 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:03:12.688947 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-bd1304b6-0d18-4066-b216-2e0847a7d396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:12.878855 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1319/2609] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:03:12.897068 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07faffe-4715-4f7e-a6a9-2664a1f58652 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:12.897068 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d07faffe-4715-4f7e-a6a9-2664a1f58652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:13.033464 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07faffe-4715-4f7e-a6a9-2664a1f58652 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:13.042924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c8dc3-06cd-47a5-a6f2-56660f581d5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:13.043956 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c8dc3-06cd-47a5-a6f2-56660f581d5b service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:13.044238 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c8dc3-06cd-47a5-a6f2-56660f581d5b service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:13.045651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c8dc3-06cd-47a5-a6f2-56660f581d5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:13.065224 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c8dc3-06cd-47a5-a6f2-56660f581d5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:13.066946 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c8dc3-06cd-47a5-a6f2-56660f581d5b service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:13.083588 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1291/2610] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:03:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:03:13.666726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01f3ee-b2d1-46a4-b99e-783ee57fde34 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:13.669614 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01f3ee-b2d1-46a4-b99e-783ee57fde34 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:13.669614 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01f3ee-b2d1-46a4-b99e-783ee57fde34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:13.708069 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-1e01f3ee-b2d1-46a4-b99e-783ee57fde34 None None] MFA Rules not processed for user `c323addb0b2d45ffb8568d6ebfb4a9eb`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:03:13.710779 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1e01f3ee-b2d1-46a4-b99e-783ee57fde34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:13.845525 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1320/2611] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:03:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:03:22.575701 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee88c9-afd0-4fa9-9a96-21eebcea34f0 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:22.576637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee88c9-afd0-4fa9-9a96-21eebcea34f0 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:22.576906 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee88c9-afd0-4fa9-9a96-21eebcea34f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:22.597709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-62ee88c9-afd0-4fa9-9a96-21eebcea34f0 None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:03:22.599813 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-62ee88c9-afd0-4fa9-9a96-21eebcea34f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:22.672663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1292/2612] 10.222.0.27 () {58 vars in 967 bytes} [Tue Aug 30 15:03:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:03:22.686942 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01f3ee-b2d1-46a4-b99e-783ee57fde34 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:22.688456 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1e01f3ee-b2d1-46a4-b99e-783ee57fde34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:22.705814 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:03:22.775235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01f3ee-b2d1-46a4-b99e-783ee57fde34 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:22.781525 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e67408-fb3e-4c81-981d-8923618118e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:22.785619 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e67408-fb3e-4c81-981d-8923618118e6 service swift] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:22.785619 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e67408-fb3e-4c81-981d-8923618118e6 service swift] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:22.785619 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e67408-fb3e-4c81-981d-8923618118e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:22.792852 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e67408-fb3e-4c81-981d-8923618118e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:22.794535 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e67408-fb3e-4c81-981d-8923618118e6 service swift] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:22.805475 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1321/2613] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 15:03:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:03:23.237307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96d9a8-b486-4917-958b-8e6817e754aa None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:23.237820 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96d9a8-b486-4917-958b-8e6817e754aa None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:23.238190 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96d9a8-b486-4917-958b-8e6817e754aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:23.268864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ec96d9a8-b486-4917-958b-8e6817e754aa None None] MFA Rules not processed for user `6a9784d49f8d45e38f096de3b2401d36`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:03:23.270138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ec96d9a8-b486-4917-958b-8e6817e754aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:23.376045 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1293/2614] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:03:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:03:23.391822 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e67408-fb3e-4c81-981d-8923618118e6 service swift] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:23.393531 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-09e67408-fb3e-4c81-981d-8923618118e6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:23.512767 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e67408-fb3e-4c81-981d-8923618118e6 service swift] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:23.520257 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cfde0-07e2-439d-a1be-a23314855842 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:23.520912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cfde0-07e2-439d-a1be-a23314855842 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:23.521181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cfde0-07e2-439d-a1be-a23314855842 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:23.521413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cfde0-07e2-439d-a1be-a23314855842 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:23.535619 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6cfde0-07e2-439d-a1be-a23314855842 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:23.537216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6cfde0-07e2-439d-a1be-a23314855842 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:23.555312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1322/2615] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:03:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:03:27.181989 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856d93c-0086-41d8-a81a-09d3e92c3e8f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:27.182862 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856d93c-0086-41d8-a81a-09d3e92c3e8f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:27.183195 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856d93c-0086-41d8-a81a-09d3e92c3e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:27.205724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e856d93c-0086-41d8-a81a-09d3e92c3e8f None None] MFA Rules not processed for user `6a9784d49f8d45e38f096de3b2401d36`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:03:27.207138 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e856d93c-0086-41d8-a81a-09d3e92c3e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:27.221406 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1294/2616] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:03:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:03:29.625696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cfde0-07e2-439d-a1be-a23314855842 service nova] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:29.627380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0c6cfde0-07e2-439d-a1be-a23314855842 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:29.641578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6cfde0-07e2-439d-a1be-a23314855842 service nova] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:29.646253 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f08a29-132d-4a48-8572-ea85ca0a32f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:29.646867 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f08a29-132d-4a48-8572-ea85ca0a32f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:29.647178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f08a29-132d-4a48-8572-ea85ca0a32f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:29.647457 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f08a29-132d-4a48-8572-ea85ca0a32f1 admin admin] PATH_INFO: `/v3/users/46ed5dde54a74913b501373fb57c25cb` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:29.648880 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f08a29-132d-4a48-8572-ea85ca0a32f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46ed5dde54a74913b501373fb57c25cb)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:29.651480 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f08a29-132d-4a48-8572-ea85ca0a32f1 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:29.840153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-69f08a29-132d-4a48-8572-ea85ca0a32f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46ed5dde54a74913b501373fb57c25cb'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:29.851581 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-69f08a29-132d-4a48-8572-ea85ca0a32f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46ed5dde54a74913b501373fb57c25cb'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:29.863851 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-69f08a29-132d-4a48-8572-ea85ca0a32f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46ed5dde54a74913b501373fb57c25cb'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:29.864674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-69f08a29-132d-4a48-8572-ea85ca0a32f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46ed5dde54a74913b501373fb57c25cb'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:29.865362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-69f08a29-132d-4a48-8572-ea85ca0a32f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46ed5dde54a74913b501373fb57c25cb'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:29.954716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:03:29.959472 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1323/2617] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:03:29 2022] DELETE /identity/v3/users/46ed5dde54a74913b501373fb57c25cb => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:03:29.964600 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce9b47-e43b-483d-ac05-6ab7d55ab554 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:29.965124 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce9b47-e43b-483d-ac05-6ab7d55ab554 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:29.965124 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce9b47-e43b-483d-ac05-6ab7d55ab554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:29.984317 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-f4ce9b47-e43b-483d-ac05-6ab7d55ab554 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:03:29.985531 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f4ce9b47-e43b-483d-ac05-6ab7d55ab554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:30.004526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:03:30.048329 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1295/2618] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:03:30.058816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f08a29-132d-4a48-8572-ea85ca0a32f1 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:30.060740 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-69f08a29-132d-4a48-8572-ea85ca0a32f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:30.117361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f08a29-132d-4a48-8572-ea85ca0a32f1 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:30.121034 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69142f-7554-4545-b8ea-bc207d0bb4bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:30.121636 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69142f-7554-4545-b8ea-bc207d0bb4bc service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:30.121916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69142f-7554-4545-b8ea-bc207d0bb4bc service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:30.122187 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69142f-7554-4545-b8ea-bc207d0bb4bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:30.131256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69142f-7554-4545-b8ea-bc207d0bb4bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:30.132734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69142f-7554-4545-b8ea-bc207d0bb4bc service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:30.142339 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1324/2619] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:03:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:03:30.578458 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce9b47-e43b-483d-ac05-6ab7d55ab554 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:30.580865 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f4ce9b47-e43b-483d-ac05-6ab7d55ab554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:30.596477 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce9b47-e43b-483d-ac05-6ab7d55ab554 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:30.603222 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd5f46-288b-43d7-88d2-79d3e9ae8f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:30.603930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd5f46-288b-43d7-88d2-79d3e9ae8f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:30.604203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd5f46-288b-43d7-88d2-79d3e9ae8f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:30.604473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd5f46-288b-43d7-88d2-79d3e9ae8f4e admin admin] PATH_INFO: `/v3/projects/1279187a4abc458fb2d4fc14375504cf` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:30.605615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd5f46-288b-43d7-88d2-79d3e9ae8f4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1279187a4abc458fb2d4fc14375504cf)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:30.607657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd5f46-288b-43d7-88d2-79d3e9ae8f4e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:30.722465 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0efd5f46-288b-43d7-88d2-79d3e9ae8f4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1279187a4abc458fb2d4fc14375504cf', 'request_id': 'req-0efd5f46-288b-43d7-88d2-79d3e9ae8f4e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:30.729737 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0efd5f46-288b-43d7-88d2-79d3e9ae8f4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1279187a4abc458fb2d4fc14375504cf', 'request_id': 'req-0efd5f46-288b-43d7-88d2-79d3e9ae8f4e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:30.736443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0efd5f46-288b-43d7-88d2-79d3e9ae8f4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1279187a4abc458fb2d4fc14375504cf', 'request_id': 'req-0efd5f46-288b-43d7-88d2-79d3e9ae8f4e'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:30.843180 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0efd5f46-288b-43d7-88d2-79d3e9ae8f4e admin admin] The token cache is being invalidate because project 1279187a4abc458fb2d4fc14375504cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:03:30.843455 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0efd5f46-288b-43d7-88d2-79d3e9ae8f4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:30.844108 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0efd5f46-288b-43d7-88d2-79d3e9ae8f4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:30.845593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1296/2620] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:03:30 2022] DELETE /identity/v3/projects/1279187a4abc458fb2d4fc14375504cf => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:03:30.851178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace4268-fe96-41fc-9aea-13722263c654 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:30.851648 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace4268-fe96-41fc-9aea-13722263c654 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:30.851818 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace4268-fe96-41fc-9aea-13722263c654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:30.873412 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-6ace4268-fe96-41fc-9aea-13722263c654 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:03:30.875285 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6ace4268-fe96-41fc-9aea-13722263c654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:30.936266 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1325/2621] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:03:30.945689 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474794ea-2494-4498-9b6f-706fa8f551a5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:30.945969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474794ea-2494-4498-9b6f-706fa8f551a5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:30.946374 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474794ea-2494-4498-9b6f-706fa8f551a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:30.968599 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-474794ea-2494-4498-9b6f-706fa8f551a5 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:03:30.970868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-474794ea-2494-4498-9b6f-706fa8f551a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:30.983479 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1297/2622] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:03:30.985640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace4268-fe96-41fc-9aea-13722263c654 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:30.996047 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace4268-fe96-41fc-9aea-13722263c654 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:31.000953 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b609b5-a0cd-4364-b8de-e43301e6bbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:31.001559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b609b5-a0cd-4364-b8de-e43301e6bbb7 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:31.001868 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b609b5-a0cd-4364-b8de-e43301e6bbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:31.002205 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b609b5-a0cd-4364-b8de-e43301e6bbb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:31.002866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b609b5-a0cd-4364-b8de-e43301e6bbb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:03:31.003199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b609b5-a0cd-4364-b8de-e43301e6bbb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:31.004803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b609b5-a0cd-4364-b8de-e43301e6bbb7 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:31.017465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1326/2623] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:03:31.022444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474794ea-2494-4498-9b6f-706fa8f551a5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:31.032004 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474794ea-2494-4498-9b6f-706fa8f551a5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:31.036936 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3e427-3339-4ea7-ab0c-51ce965f1d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:31.037532 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3e427-3339-4ea7-ab0c-51ce965f1d1d admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:31.037812 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3e427-3339-4ea7-ab0c-51ce965f1d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:31.038083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3e427-3339-4ea7-ab0c-51ce965f1d1d admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:31.038723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3e427-3339-4ea7-ab0c-51ce965f1d1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:31.040691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3e427-3339-4ea7-ab0c-51ce965f1d1d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:31.166149 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1298/2624] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:03:31 2022] POST /identity/v3/projects => generated 389 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:03:31.169991 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b609b5-a0cd-4364-b8de-e43301e6bbb7 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:31.180866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b609b5-a0cd-4364-b8de-e43301e6bbb7 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:31.256055 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70942e50-4a56-4196-b30f-7565153b711d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:31.256708 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70942e50-4a56-4196-b30f-7565153b711d admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:31.256979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70942e50-4a56-4196-b30f-7565153b711d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:31.257244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70942e50-4a56-4196-b30f-7565153b711d admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:31.257893 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70942e50-4a56-4196-b30f-7565153b711d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:31.259856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70942e50-4a56-4196-b30f-7565153b711d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:31.366550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1327/2625] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:03:31 2022] POST /identity/v3/users => generated 295 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:03:31.371130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3e427-3339-4ea7-ab0c-51ce965f1d1d admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:31.394199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3e427-3339-4ea7-ab0c-51ce965f1d1d admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:31.413677 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8387398-35b6-4f36-a789-a4f2eafbb083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:31.414433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8387398-35b6-4f36-a789-a4f2eafbb083 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:31.414782 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8387398-35b6-4f36-a789-a4f2eafbb083 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:31.415144 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8387398-35b6-4f36-a789-a4f2eafbb083 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:31.416478 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8387398-35b6-4f36-a789-a4f2eafbb083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:31.418250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8387398-35b6-4f36-a789-a4f2eafbb083 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:31.429481 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1299/2626] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:03:31.434868 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70942e50-4a56-4196-b30f-7565153b711d admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:31.447169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70942e50-4a56-4196-b30f-7565153b711d admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:31.452247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7a5a0-e337-4d43-ae54-204354f5430b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:31.452919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7a5a0-e337-4d43-ae54-204354f5430b admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:31.454897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7a5a0-e337-4d43-ae54-204354f5430b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:31.454897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7a5a0-e337-4d43-ae54-204354f5430b admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:31.454897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7a5a0-e337-4d43-ae54-204354f5430b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:31.455806 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7a5a0-e337-4d43-ae54-204354f5430b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:31.466996 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1328/2627] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:03:31.471487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8387398-35b6-4f36-a789-a4f2eafbb083 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:31.487315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8387398-35b6-4f36-a789-a4f2eafbb083 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:31.489690 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd208a7a-86f8-4d13-bc46-030646565558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:31.490343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd208a7a-86f8-4d13-bc46-030646565558 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:31.490618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd208a7a-86f8-4d13-bc46-030646565558 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:31.490888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd208a7a-86f8-4d13-bc46-030646565558 admin admin] PATH_INFO: `/v3/projects/c6fc2cf7b91b4348b144506a97f27b13/users/ba05f742b02f476e8d078989abf416ce/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:31.507857 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd208a7a-86f8-4d13-bc46-030646565558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6fc2cf7b91b4348b144506a97f27b13, user_id=ba05f742b02f476e8d078989abf416ce, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:31.510622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd208a7a-86f8-4d13-bc46-030646565558 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:31.600069 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1300/2628] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:03:31 2022] PUT /identity/v3/projects/c6fc2cf7b91b4348b144506a97f27b13/users/ba05f742b02f476e8d078989abf416ce/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:03:31.611527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5c39a-0878-4f1c-9882-c9f582ce785b None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:31.613328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5c39a-0878-4f1c-9882-c9f582ce785b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:31.613615 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5c39a-0878-4f1c-9882-c9f582ce785b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:31.634044 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:03:31.645142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-51e5c39a-0878-4f1c-9882-c9f582ce785b None None] MFA Rules not processed for user `ba05f742b02f476e8d078989abf416ce`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:03:31.646381 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-51e5c39a-0878-4f1c-9882-c9f582ce785b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:31.714205 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1329/2629] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:03:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:03:31.724964 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd208a7a-86f8-4d13-bc46-030646565558 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:31.726755 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bd208a7a-86f8-4d13-bc46-030646565558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:31.785982 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:03:31.802267 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd208a7a-86f8-4d13-bc46-030646565558 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:31.807016 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f073db6-3150-4286-a133-08afaf2c5891 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:31.807856 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f073db6-3150-4286-a133-08afaf2c5891 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:31.808316 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f073db6-3150-4286-a133-08afaf2c5891 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:31.808745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f073db6-3150-4286-a133-08afaf2c5891 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:31.822718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f073db6-3150-4286-a133-08afaf2c5891 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:31.824966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f073db6-3150-4286-a133-08afaf2c5891 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:31.846083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1301/2630] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:03:31 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:03:41.920508 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=28 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:03:42.753340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:03:49.762716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5c39a-0878-4f1c-9882-c9f582ce785b None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:49.764674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-51e5c39a-0878-4f1c-9882-c9f582ce785b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:49.863133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5c39a-0878-4f1c-9882-c9f582ce785b None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:49.871665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755f840-f7fb-4878-8e9a-92900709ed49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:49.877666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755f840-f7fb-4878-8e9a-92900709ed49 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:49.877666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755f840-f7fb-4878-8e9a-92900709ed49 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:49.877666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755f840-f7fb-4878-8e9a-92900709ed49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:49.877666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-f755f840-f7fb-4878-8e9a-92900709ed49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:49.889914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755f840-f7fb-4878-8e9a-92900709ed49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:49.891881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f755f840-f7fb-4878-8e9a-92900709ed49 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:49.904056 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1330/2631] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:03:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:03:50.054770 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f073db6-3150-4286-a133-08afaf2c5891 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:50.057034 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7f073db6-3150-4286-a133-08afaf2c5891 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:50.125093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f073db6-3150-4286-a133-08afaf2c5891 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:50.128831 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4191db-01c9-42b1-85fa-36a5e6b1bc9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:50.129440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4191db-01c9-42b1-85fa-36a5e6b1bc9f service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:50.129720 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4191db-01c9-42b1-85fa-36a5e6b1bc9f service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:50.129990 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4191db-01c9-42b1-85fa-36a5e6b1bc9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:50.131890 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8c4191db-01c9-42b1-85fa-36a5e6b1bc9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:50.207943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4191db-01c9-42b1-85fa-36a5e6b1bc9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:50.209554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4191db-01c9-42b1-85fa-36a5e6b1bc9f service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:50.221659 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1302/2632] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:03:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:03:53.486802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755f840-f7fb-4878-8e9a-92900709ed49 service nova] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:53.577508 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755f840-f7fb-4878-8e9a-92900709ed49 service nova] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:53.582321 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f1fff-10f4-4341-8906-5802b476349c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:53.583062 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f1fff-10f4-4341-8906-5802b476349c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:53.583457 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f1fff-10f4-4341-8906-5802b476349c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:53.583828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f1fff-10f4-4341-8906-5802b476349c admin admin] PATH_INFO: `/v3/users/ba05f742b02f476e8d078989abf416ce` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:53.585085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f1fff-10f4-4341-8906-5802b476349c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba05f742b02f476e8d078989abf416ce)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:53.592878 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f1fff-10f4-4341-8906-5802b476349c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:53.865936 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0d0f1fff-10f4-4341-8906-5802b476349c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba05f742b02f476e8d078989abf416ce'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:53.874215 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0d0f1fff-10f4-4341-8906-5802b476349c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba05f742b02f476e8d078989abf416ce'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:53.891635 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0d0f1fff-10f4-4341-8906-5802b476349c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba05f742b02f476e8d078989abf416ce'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:53.894691 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0d0f1fff-10f4-4341-8906-5802b476349c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba05f742b02f476e8d078989abf416ce'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:53.895456 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-0d0f1fff-10f4-4341-8906-5802b476349c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba05f742b02f476e8d078989abf416ce'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:53.993246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1331/2633] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:03:53 2022] DELETE /identity/v3/users/ba05f742b02f476e8d078989abf416ce => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:03:53.998573 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66203815-187f-42bf-a2f9-62d31f331e44 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:53.999079 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66203815-187f-42bf-a2f9-62d31f331e44 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:53.999226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66203815-187f-42bf-a2f9-62d31f331e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:54.018940 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-66203815-187f-42bf-a2f9-62d31f331e44 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:03:54.020475 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-66203815-187f-42bf-a2f9-62d31f331e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:54.082147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1303/2634] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:03:54.092656 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f1fff-10f4-4341-8906-5802b476349c admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:54.094094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0d0f1fff-10f4-4341-8906-5802b476349c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:54.101147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:03:54.157068 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f1fff-10f4-4341-8906-5802b476349c admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:54.160793 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144898e7-5154-403d-8347-7dae6ee4fae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:54.161395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144898e7-5154-403d-8347-7dae6ee4fae8 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:54.161654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144898e7-5154-403d-8347-7dae6ee4fae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:54.161918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144898e7-5154-403d-8347-7dae6ee4fae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:54.171442 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144898e7-5154-403d-8347-7dae6ee4fae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:54.172868 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144898e7-5154-403d-8347-7dae6ee4fae8 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:54.182109 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1332/2635] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:03:54.628861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66203815-187f-42bf-a2f9-62d31f331e44 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:54.630549 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-66203815-187f-42bf-a2f9-62d31f331e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:54.644846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66203815-187f-42bf-a2f9-62d31f331e44 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:54.649439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffee908-9ecc-4473-9d7c-8e1fae9ae138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:54.654605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffee908-9ecc-4473-9d7c-8e1fae9ae138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:54.654605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffee908-9ecc-4473-9d7c-8e1fae9ae138 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:54.654605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffee908-9ecc-4473-9d7c-8e1fae9ae138 admin admin] PATH_INFO: `/v3/projects/c6fc2cf7b91b4348b144506a97f27b13` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:54.654605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffee908-9ecc-4473-9d7c-8e1fae9ae138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6fc2cf7b91b4348b144506a97f27b13)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:54.654605 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffee908-9ecc-4473-9d7c-8e1fae9ae138 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:54.745637 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:03:54.748244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-dffee908-9ecc-4473-9d7c-8e1fae9ae138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6fc2cf7b91b4348b144506a97f27b13', 'request_id': 'req-dffee908-9ecc-4473-9d7c-8e1fae9ae138'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:54.754224 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-dffee908-9ecc-4473-9d7c-8e1fae9ae138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6fc2cf7b91b4348b144506a97f27b13', 'request_id': 'req-dffee908-9ecc-4473-9d7c-8e1fae9ae138'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:54.760327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-dffee908-9ecc-4473-9d7c-8e1fae9ae138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6fc2cf7b91b4348b144506a97f27b13', 'request_id': 'req-dffee908-9ecc-4473-9d7c-8e1fae9ae138'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:54.837728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-dffee908-9ecc-4473-9d7c-8e1fae9ae138 admin admin] The token cache is being invalidate because project c6fc2cf7b91b4348b144506a97f27b13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:03:54.838303 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-dffee908-9ecc-4473-9d7c-8e1fae9ae138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:54.838930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-dffee908-9ecc-4473-9d7c-8e1fae9ae138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:03:54.840429 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1304/2636] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:03:54 2022] DELETE /identity/v3/projects/c6fc2cf7b91b4348b144506a97f27b13 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:04.290481 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=24 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:04.861315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:13.133608 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144898e7-5154-403d-8347-7dae6ee4fae8 service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:13.142716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144898e7-5154-403d-8347-7dae6ee4fae8 service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:13.190922 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6e115-a47f-4222-8987-5c441e9a9d03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:13.191550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6e115-a47f-4222-8987-5c441e9a9d03 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:13.191827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6e115-a47f-4222-8987-5c441e9a9d03 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:13.192097 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6e115-a47f-4222-8987-5c441e9a9d03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:13.193831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8ad6e115-a47f-4222-8987-5c441e9a9d03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:13.256097 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad6e115-a47f-4222-8987-5c441e9a9d03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:13.257532 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad6e115-a47f-4222-8987-5c441e9a9d03 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:13.268374 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1333/2637] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:17.046538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb0afa-05ae-4280-a538-ca35df27593d None None] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:17.049416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb0afa-05ae-4280-a538-ca35df27593d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:17.049416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb0afa-05ae-4280-a538-ca35df27593d None None] PATH_INFO: `/` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:17.049416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1305/2638] 10.222.0.27 () {54 vars in 961 bytes} [Tue Aug 30 15:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:04:17.053640 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f39ee-fa55-4654-a85e-4a1a34e3fa99 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:17.054124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f39ee-fa55-4654-a85e-4a1a34e3fa99 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:17.054222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f39ee-fa55-4654-a85e-4a1a34e3fa99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:17.074163 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-b20f39ee-fa55-4654-a85e-4a1a34e3fa99 None None] MFA Rules not processed for user `f4f11655cd51438e9652869f74ffa975`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:17.075438 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b20f39ee-fa55-4654-a85e-4a1a34e3fa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:17.140128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1334/2639] 10.222.0.27 () {58 vars in 1058 bytes} [Tue Aug 30 15:04:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:17.148631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb0afa-05ae-4280-a538-ca35df27593d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:17.162557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb0afa-05ae-4280-a538-ca35df27593d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:17.167419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43e39a-70a7-4a53-aa28-ea7c34b74326 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4f11655cd51438e9652869f74ffa975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:17.168002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43e39a-70a7-4a53-aa28-ea7c34b74326 service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:17.168277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43e39a-70a7-4a53-aa28-ea7c34b74326 service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:17.168542 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43e39a-70a7-4a53-aa28-ea7c34b74326 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:17.170853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ac43e39a-70a7-4a53-aa28-ea7c34b74326 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:17.182161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43e39a-70a7-4a53-aa28-ea7c34b74326 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:17.183783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43e39a-70a7-4a53-aa28-ea7c34b74326 service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:17.196818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1306/2640] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:17.198743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f39ee-fa55-4654-a85e-4a1a34e3fa99 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:17.200192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b20f39ee-fa55-4654-a85e-4a1a34e3fa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:17.214325 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f39ee-fa55-4654-a85e-4a1a34e3fa99 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:17.219626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b37d03-0b32-4e4f-a3a4-26f2c90dc5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:17.220258 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b37d03-0b32-4e4f-a3a4-26f2c90dc5b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:17.220540 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b37d03-0b32-4e4f-a3a4-26f2c90dc5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:17.220841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b37d03-0b32-4e4f-a3a4-26f2c90dc5b2 admin admin] PATH_INFO: `/v3/users/7dc5cd44b1b34a7ebbdca2c062715431` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:17.221963 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b37d03-0b32-4e4f-a3a4-26f2c90dc5b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dc5cd44b1b34a7ebbdca2c062715431)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:17.224200 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b37d03-0b32-4e4f-a3a4-26f2c90dc5b2 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:17.389158 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:17.393975 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-04b37d03-0b32-4e4f-a3a4-26f2c90dc5b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dc5cd44b1b34a7ebbdca2c062715431'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:17.400710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-04b37d03-0b32-4e4f-a3a4-26f2c90dc5b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dc5cd44b1b34a7ebbdca2c062715431'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:17.413244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-04b37d03-0b32-4e4f-a3a4-26f2c90dc5b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dc5cd44b1b34a7ebbdca2c062715431'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:17.413967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-04b37d03-0b32-4e4f-a3a4-26f2c90dc5b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dc5cd44b1b34a7ebbdca2c062715431'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:17.414613 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-04b37d03-0b32-4e4f-a3a4-26f2c90dc5b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dc5cd44b1b34a7ebbdca2c062715431'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:17.524227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1335/2641] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:04:17 2022] DELETE /identity/v3/users/7dc5cd44b1b34a7ebbdca2c062715431 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:17.765488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43e39a-70a7-4a53-aa28-ea7c34b74326 service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:17.776301 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43e39a-70a7-4a53-aa28-ea7c34b74326 service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:17.833535 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e7e5e-6f69-4744-8e93-ff30cf92a128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:17.834160 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e7e5e-6f69-4744-8e93-ff30cf92a128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:17.834439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e7e5e-6f69-4744-8e93-ff30cf92a128 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:17.834711 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e7e5e-6f69-4744-8e93-ff30cf92a128 admin admin] PATH_INFO: `/v3/projects/af84072cdab4455a95655c3c5628e3ac` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:17.835852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e7e5e-6f69-4744-8e93-ff30cf92a128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af84072cdab4455a95655c3c5628e3ac)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:17.837831 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e7e5e-6f69-4744-8e93-ff30cf92a128 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:17.946997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8f3e7e5e-6f69-4744-8e93-ff30cf92a128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af84072cdab4455a95655c3c5628e3ac', 'request_id': 'req-8f3e7e5e-6f69-4744-8e93-ff30cf92a128'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:17.954309 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8f3e7e5e-6f69-4744-8e93-ff30cf92a128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af84072cdab4455a95655c3c5628e3ac', 'request_id': 'req-8f3e7e5e-6f69-4744-8e93-ff30cf92a128'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:17.959761 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8f3e7e5e-6f69-4744-8e93-ff30cf92a128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af84072cdab4455a95655c3c5628e3ac', 'request_id': 'req-8f3e7e5e-6f69-4744-8e93-ff30cf92a128'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:18.031564 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8f3e7e5e-6f69-4744-8e93-ff30cf92a128 admin admin] The token cache is being invalidate because project af84072cdab4455a95655c3c5628e3ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:04:18.032002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8f3e7e5e-6f69-4744-8e93-ff30cf92a128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:18.033271 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8f3e7e5e-6f69-4744-8e93-ff30cf92a128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:18.034332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1307/2642] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:04:17 2022] DELETE /identity/v3/projects/af84072cdab4455a95655c3c5628e3ac => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:18.039232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee63191-4ff3-45de-a0d2-f2bd847659b2 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:18.039523 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee63191-4ff3-45de-a0d2-f2bd847659b2 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:18.039801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee63191-4ff3-45de-a0d2-f2bd847659b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:18.060000 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-4ee63191-4ff3-45de-a0d2-f2bd847659b2 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:18.061632 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-4ee63191-4ff3-45de-a0d2-f2bd847659b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:18.127841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1336/2643] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:18.135985 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b48fc-655c-4545-99a8-b7d8f65e3e01 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:18.136480 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b48fc-655c-4545-99a8-b7d8f65e3e01 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:18.136633 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b48fc-655c-4545-99a8-b7d8f65e3e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:18.156342 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4d7b48fc-655c-4545-99a8-b7d8f65e3e01 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:18.157522 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4d7b48fc-655c-4545-99a8-b7d8f65e3e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:18.165267 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1308/2644] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:18.169478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee63191-4ff3-45de-a0d2-f2bd847659b2 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:18.179303 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee63191-4ff3-45de-a0d2-f2bd847659b2 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:18.183985 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1c511-1d05-4682-b2c3-c021a1eb1717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:18.184572 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1c511-1d05-4682-b2c3-c021a1eb1717 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:18.184870 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1c511-1d05-4682-b2c3-c021a1eb1717 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:18.185641 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1c511-1d05-4682-b2c3-c021a1eb1717 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:18.186287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1c511-1d05-4682-b2c3-c021a1eb1717 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:04:18.186591 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1c511-1d05-4682-b2c3-c021a1eb1717 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:18.188225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1c511-1d05-4682-b2c3-c021a1eb1717 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:18.199611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1337/2645] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:04:18.204198 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b48fc-655c-4545-99a8-b7d8f65e3e01 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:18.213250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b48fc-655c-4545-99a8-b7d8f65e3e01 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:18.217894 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f9c46-e705-4787-ae72-9d6051164cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:18.218479 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f9c46-e705-4787-ae72-9d6051164cbb admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:18.218759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f9c46-e705-4787-ae72-9d6051164cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:18.219030 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f9c46-e705-4787-ae72-9d6051164cbb admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:18.219699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f9c46-e705-4787-ae72-9d6051164cbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:18.221618 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f9c46-e705-4787-ae72-9d6051164cbb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:18.388118 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1309/2646] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:04:18 2022] POST /identity/v3/projects => generated 405 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:04:18.392446 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1c511-1d05-4682-b2c3-c021a1eb1717 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:18.402352 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1c511-1d05-4682-b2c3-c021a1eb1717 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:18.456995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c172-d087-4fb2-94ef-7a6727492350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:18.457588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c172-d087-4fb2-94ef-7a6727492350 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:18.457883 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c172-d087-4fb2-94ef-7a6727492350 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:18.458132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c172-d087-4fb2-94ef-7a6727492350 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:18.458749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1c172-d087-4fb2-94ef-7a6727492350 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:18.460706 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1c172-d087-4fb2-94ef-7a6727492350 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:18.540767 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1338/2647] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:04:18 2022] POST /identity/v3/users => generated 303 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:04:18.545542 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f9c46-e705-4787-ae72-9d6051164cbb admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:18.555988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f9c46-e705-4787-ae72-9d6051164cbb admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:18.561216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3f3a1-64a9-4112-bb70-f2914b175ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:18.561807 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3f3a1-64a9-4112-bb70-f2914b175ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:18.562115 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3f3a1-64a9-4112-bb70-f2914b175ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:18.562385 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3f3a1-64a9-4112-bb70-f2914b175ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:18.563022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3f3a1-64a9-4112-bb70-f2914b175ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:18.564635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3f3a1-64a9-4112-bb70-f2914b175ce1 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:18.574505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1310/2648] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:04:18.578280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c172-d087-4fb2-94ef-7a6727492350 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:18.588177 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c172-d087-4fb2-94ef-7a6727492350 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:18.592784 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ee2c1-5d2f-4d31-82d0-f7dee673680e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:18.593373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ee2c1-5d2f-4d31-82d0-f7dee673680e admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:18.593649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ee2c1-5d2f-4d31-82d0-f7dee673680e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:18.593918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ee2c1-5d2f-4d31-82d0-f7dee673680e admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:18.594556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ee2c1-5d2f-4d31-82d0-f7dee673680e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:18.596165 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ee2c1-5d2f-4d31-82d0-f7dee673680e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:18.606810 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1339/2649] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:04:18.611379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3f3a1-64a9-4112-bb70-f2914b175ce1 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:18.621216 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3f3a1-64a9-4112-bb70-f2914b175ce1 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:18.626530 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac68510-ab1f-49e6-9d45-f0136c0c896c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:18.627186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac68510-ab1f-49e6-9d45-f0136c0c896c admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:18.627469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac68510-ab1f-49e6-9d45-f0136c0c896c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:18.627753 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac68510-ab1f-49e6-9d45-f0136c0c896c admin admin] PATH_INFO: `/v3/projects/6f49adf3f705487bb7ec2b0c6dce7fb2/users/b3e225fa6bea4292adb21b28a89fd3f7/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:18.641803 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac68510-ab1f-49e6-9d45-f0136c0c896c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f49adf3f705487bb7ec2b0c6dce7fb2, user_id=b3e225fa6bea4292adb21b28a89fd3f7, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:18.644411 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac68510-ab1f-49e6-9d45-f0136c0c896c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:18.712519 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1311/2650] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:04:18 2022] PUT /identity/v3/projects/6f49adf3f705487bb7ec2b0c6dce7fb2/users/b3e225fa6bea4292adb21b28a89fd3f7/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:18.718236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f4f13-8c6a-497b-855c-2fd3a0ec74ac None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:18.718522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f4f13-8c6a-497b-855c-2fd3a0ec74ac None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:18.718797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f4f13-8c6a-497b-855c-2fd3a0ec74ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:18.739195 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-a36f4f13-8c6a-497b-855c-2fd3a0ec74ac None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:18.740410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a36f4f13-8c6a-497b-855c-2fd3a0ec74ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:18.759802 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:18.803332 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1340/2651] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:18.816657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac68510-ab1f-49e6-9d45-f0136c0c896c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:18.816657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-dac68510-ab1f-49e6-9d45-f0136c0c896c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:18.829926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:18.873204 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac68510-ab1f-49e6-9d45-f0136c0c896c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:18.877416 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb679fe-389b-4690-9d0d-2126e6ed13c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:18.878020 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb679fe-389b-4690-9d0d-2126e6ed13c5 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:18.878289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb679fe-389b-4690-9d0d-2126e6ed13c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:18.878630 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb679fe-389b-4690-9d0d-2126e6ed13c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:18.887473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb679fe-389b-4690-9d0d-2126e6ed13c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:18.888887 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb679fe-389b-4690-9d0d-2126e6ed13c5 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:18.898756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1312/2652] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:20.416602 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f283e50-004a-48f8-9b8d-2bb32a57e554 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:20.417140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f283e50-004a-48f8-9b8d-2bb32a57e554 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:20.417243 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f283e50-004a-48f8-9b8d-2bb32a57e554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:20.448035 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-0f283e50-004a-48f8-9b8d-2bb32a57e554 None None] MFA Rules not processed for user `b3e225fa6bea4292adb21b28a89fd3f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:20.449295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0f283e50-004a-48f8-9b8d-2bb32a57e554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:20.518807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1341/2653] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:20 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:20.523535 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb679fe-389b-4690-9d0d-2126e6ed13c5 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:20.534161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb679fe-389b-4690-9d0d-2126e6ed13c5 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:20.538676 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf468ea-8a51-4009-838d-755ef0d53b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:20.539322 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf468ea-8a51-4009-838d-755ef0d53b86 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:20.539601 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf468ea-8a51-4009-838d-755ef0d53b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:20.539867 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf468ea-8a51-4009-838d-755ef0d53b86 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:20.540483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf468ea-8a51-4009-838d-755ef0d53b86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:20.542372 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf468ea-8a51-4009-838d-755ef0d53b86 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:20.622345 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1313/2654] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:04:20 2022] POST /identity/v3/projects => generated 401 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:04:20.626415 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f283e50-004a-48f8-9b8d-2bb32a57e554 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:20.637280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f283e50-004a-48f8-9b8d-2bb32a57e554 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:20.693904 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa434f1b-8e8f-40d0-840d-6c383abccdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:20.694408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa434f1b-8e8f-40d0-840d-6c383abccdc6 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:20.694684 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa434f1b-8e8f-40d0-840d-6c383abccdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:20.694953 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa434f1b-8e8f-40d0-840d-6c383abccdc6 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:20.695677 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa434f1b-8e8f-40d0-840d-6c383abccdc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:20.697572 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa434f1b-8e8f-40d0-840d-6c383abccdc6 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:20.803848 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1342/2655] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:04:20 2022] POST /identity/v3/users => generated 307 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:04:20.807668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf468ea-8a51-4009-838d-755ef0d53b86 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:20.818053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf468ea-8a51-4009-838d-755ef0d53b86 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:20.822596 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49628a-4e18-4dc6-a04a-ce4811950a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:20.823215 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49628a-4e18-4dc6-a04a-ce4811950a47 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:20.823490 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49628a-4e18-4dc6-a04a-ce4811950a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:20.823756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49628a-4e18-4dc6-a04a-ce4811950a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:20.824432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d49628a-4e18-4dc6-a04a-ce4811950a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:20.826505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d49628a-4e18-4dc6-a04a-ce4811950a47 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:20.837017 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1314/2656] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:04:20.840992 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa434f1b-8e8f-40d0-840d-6c383abccdc6 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:20.851758 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa434f1b-8e8f-40d0-840d-6c383abccdc6 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:20.856367 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993171a8-6768-4b47-bd43-5073a392dcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:20.856997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993171a8-6768-4b47-bd43-5073a392dcf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:20.857287 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993171a8-6768-4b47-bd43-5073a392dcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:20.857544 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993171a8-6768-4b47-bd43-5073a392dcf0 admin admin] PATH_INFO: `/v3/projects/c75b28e4a94b43c197bc5a418b00c4dd/users/327c7b1a976a4a9aa3416dc410294279/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:20.871829 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993171a8-6768-4b47-bd43-5073a392dcf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c75b28e4a94b43c197bc5a418b00c4dd, user_id=327c7b1a976a4a9aa3416dc410294279, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:20.874955 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993171a8-6768-4b47-bd43-5073a392dcf0 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:20.934733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1343/2657] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:04:20 2022] PUT /identity/v3/projects/c75b28e4a94b43c197bc5a418b00c4dd/users/327c7b1a976a4a9aa3416dc410294279/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:20.939161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49628a-4e18-4dc6-a04a-ce4811950a47 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:20.948994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49628a-4e18-4dc6-a04a-ce4811950a47 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:21.004173 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78deb77b-a247-4a35-b65e-ce412a57ea5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:21.004802 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78deb77b-a247-4a35-b65e-ce412a57ea5c admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:21.005076 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78deb77b-a247-4a35-b65e-ce412a57ea5c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:21.005344 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78deb77b-a247-4a35-b65e-ce412a57ea5c admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:21.005984 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78deb77b-a247-4a35-b65e-ce412a57ea5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:21.007596 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78deb77b-a247-4a35-b65e-ce412a57ea5c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:21.017843 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1315/2658] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:04:21.021975 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993171a8-6768-4b47-bd43-5073a392dcf0 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:21.032452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993171a8-6768-4b47-bd43-5073a392dcf0 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:21.037243 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c706e-d5dd-4b72-856c-aaf1394cd077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:21.037894 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c706e-d5dd-4b72-856c-aaf1394cd077 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:21.038171 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c706e-d5dd-4b72-856c-aaf1394cd077 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:21.038441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c706e-d5dd-4b72-856c-aaf1394cd077 admin admin] PATH_INFO: `/v3/projects/c75b28e4a94b43c197bc5a418b00c4dd/users/327c7b1a976a4a9aa3416dc410294279/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:21.040596 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c706e-d5dd-4b72-856c-aaf1394cd077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c75b28e4a94b43c197bc5a418b00c4dd, user_id=327c7b1a976a4a9aa3416dc410294279, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:21.043365 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c706e-d5dd-4b72-856c-aaf1394cd077 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:21.054108 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1344/2659] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:04:21 2022] PUT /identity/v3/projects/c75b28e4a94b43c197bc5a418b00c4dd/users/327c7b1a976a4a9aa3416dc410294279/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:22.351898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa5067-80c8-48af-8004-b4ec652ea04b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:22.352507 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa5067-80c8-48af-8004-b4ec652ea04b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:22.352615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa5067-80c8-48af-8004-b4ec652ea04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:22.382955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-faaa5067-80c8-48af-8004-b4ec652ea04b None None] MFA Rules not processed for user `327c7b1a976a4a9aa3416dc410294279`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:22.384374 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-faaa5067-80c8-48af-8004-b4ec652ea04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:22.457913 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1316/2660] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:22 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:23.798964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c706e-d5dd-4b72-856c-aaf1394cd077 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:23.808408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:23.863152 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c706e-d5dd-4b72-856c-aaf1394cd077 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:23.867732 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29f4c0-4ae8-4407-938d-fff739590430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:23.868353 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29f4c0-4ae8-4407-938d-fff739590430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:23.868631 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29f4c0-4ae8-4407-938d-fff739590430 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:23.868902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29f4c0-4ae8-4407-938d-fff739590430 admin admin] PATH_INFO: `/v3/users/b3e225fa6bea4292adb21b28a89fd3f7` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:23.870010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29f4c0-4ae8-4407-938d-fff739590430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3e225fa6bea4292adb21b28a89fd3f7)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:23.871972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29f4c0-4ae8-4407-938d-fff739590430 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:24.085666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7b29f4c0-4ae8-4407-938d-fff739590430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3e225fa6bea4292adb21b28a89fd3f7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:24.093071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7b29f4c0-4ae8-4407-938d-fff739590430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3e225fa6bea4292adb21b28a89fd3f7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:24.104997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7b29f4c0-4ae8-4407-938d-fff739590430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3e225fa6bea4292adb21b28a89fd3f7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:24.105667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7b29f4c0-4ae8-4407-938d-fff739590430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3e225fa6bea4292adb21b28a89fd3f7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:24.106274 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7b29f4c0-4ae8-4407-938d-fff739590430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3e225fa6bea4292adb21b28a89fd3f7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:24.157960 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1345/2661] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:04:23 2022] DELETE /identity/v3/users/b3e225fa6bea4292adb21b28a89fd3f7 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:24.393562 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa5067-80c8-48af-8004-b4ec652ea04b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:24.400693 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:24.405842 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa5067-80c8-48af-8004-b4ec652ea04b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:24.463005 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:24.463648 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:24.463926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:24.464195 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714 admin admin] PATH_INFO: `/v3/projects/6f49adf3f705487bb7ec2b0c6dce7fb2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:24.465439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f49adf3f705487bb7ec2b0c6dce7fb2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:24.468947 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:24.574234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f49adf3f705487bb7ec2b0c6dce7fb2', 'request_id': 'req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:24.580782 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f49adf3f705487bb7ec2b0c6dce7fb2', 'request_id': 'req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:24.586278 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f49adf3f705487bb7ec2b0c6dce7fb2', 'request_id': 'req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:24.683928 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714 admin admin] The token cache is being invalidate because project 6f49adf3f705487bb7ec2b0c6dce7fb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:04:24.684183 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:24.684816 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:24.686846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1317/2662] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:04:24 2022] DELETE /identity/v3/projects/6f49adf3f705487bb7ec2b0c6dce7fb2 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:24.690837 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29f4c0-4ae8-4407-938d-fff739590430 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:24.692310 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7b29f4c0-4ae8-4407-938d-fff739590430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:24.706183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29f4c0-4ae8-4407-938d-fff739590430 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:24.762521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a8285-7801-43a7-aa07-df48a5b5ee52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:24.763175 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a8285-7801-43a7-aa07-df48a5b5ee52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:24.763457 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a8285-7801-43a7-aa07-df48a5b5ee52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:24.763868 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a8285-7801-43a7-aa07-df48a5b5ee52 admin admin] PATH_INFO: `/v3/users/327c7b1a976a4a9aa3416dc410294279` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:24.766041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a8285-7801-43a7-aa07-df48a5b5ee52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=327c7b1a976a4a9aa3416dc410294279)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:24.768054 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a8285-7801-43a7-aa07-df48a5b5ee52 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:24.885436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:25.006175 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e23a8285-7801-43a7-aa07-df48a5b5ee52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '327c7b1a976a4a9aa3416dc410294279'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:25.013092 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e23a8285-7801-43a7-aa07-df48a5b5ee52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '327c7b1a976a4a9aa3416dc410294279'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:25.026130 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e23a8285-7801-43a7-aa07-df48a5b5ee52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '327c7b1a976a4a9aa3416dc410294279'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:25.026860 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e23a8285-7801-43a7-aa07-df48a5b5ee52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '327c7b1a976a4a9aa3416dc410294279'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:25.027496 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-e23a8285-7801-43a7-aa07-df48a5b5ee52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '327c7b1a976a4a9aa3416dc410294279'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:25.140885 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1346/2663] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:04:24 2022] DELETE /identity/v3/users/327c7b1a976a4a9aa3416dc410294279 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:25.445200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:25.446610 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:25.459932 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9c6f9-cb32-4f44-85f2-92aa9bd83714 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:25.517435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a357d46-e3aa-4272-b75f-f0320062c879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:25.518086 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a357d46-e3aa-4272-b75f-f0320062c879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:25.518363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a357d46-e3aa-4272-b75f-f0320062c879 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:25.518633 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a357d46-e3aa-4272-b75f-f0320062c879 admin admin] PATH_INFO: `/v3/projects/c75b28e4a94b43c197bc5a418b00c4dd` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:25.519830 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a357d46-e3aa-4272-b75f-f0320062c879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c75b28e4a94b43c197bc5a418b00c4dd)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:25.521726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a357d46-e3aa-4272-b75f-f0320062c879 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:25.624157 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:25.632525 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7a357d46-e3aa-4272-b75f-f0320062c879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c75b28e4a94b43c197bc5a418b00c4dd', 'request_id': 'req-7a357d46-e3aa-4272-b75f-f0320062c879'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:25.638728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7a357d46-e3aa-4272-b75f-f0320062c879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c75b28e4a94b43c197bc5a418b00c4dd', 'request_id': 'req-7a357d46-e3aa-4272-b75f-f0320062c879'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:25.644910 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7a357d46-e3aa-4272-b75f-f0320062c879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c75b28e4a94b43c197bc5a418b00c4dd', 'request_id': 'req-7a357d46-e3aa-4272-b75f-f0320062c879'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:25.716815 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7a357d46-e3aa-4272-b75f-f0320062c879 admin admin] The token cache is being invalidate because project c75b28e4a94b43c197bc5a418b00c4dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:04:25.717219 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7a357d46-e3aa-4272-b75f-f0320062c879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:25.717859 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7a357d46-e3aa-4272-b75f-f0320062c879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:25.719459 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1318/2664] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:04:25 2022] DELETE /identity/v3/projects/c75b28e4a94b43c197bc5a418b00c4dd => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:25.724877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013418c-2b75-4e3d-9461-a74264572bcd None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:25.725163 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013418c-2b75-4e3d-9461-a74264572bcd None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:25.725440 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013418c-2b75-4e3d-9461-a74264572bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:25.745744 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-e013418c-2b75-4e3d-9461-a74264572bcd None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:25.752050 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e013418c-2b75-4e3d-9461-a74264572bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:25.810578 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1347/2665] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:04:25.819053 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d36c6-67a1-4332-918b-d5ab3d8a16f5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:25.819440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d36c6-67a1-4332-918b-d5ab3d8a16f5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:25.819723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d36c6-67a1-4332-918b-d5ab3d8a16f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:25.839456 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-d72d36c6-67a1-4332-918b-d5ab3d8a16f5 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:25.840667 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d72d36c6-67a1-4332-918b-d5ab3d8a16f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:25.848967 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1319/2666] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:25.853527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013418c-2b75-4e3d-9461-a74264572bcd None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:25.863954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013418c-2b75-4e3d-9461-a74264572bcd None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:25.868646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa32399-b628-4f44-962f-4afb454a76cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:25.869361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa32399-b628-4f44-962f-4afb454a76cd admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:25.869641 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa32399-b628-4f44-962f-4afb454a76cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:25.869915 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa32399-b628-4f44-962f-4afb454a76cd admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:25.870567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa32399-b628-4f44-962f-4afb454a76cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:04:25.870854 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa32399-b628-4f44-962f-4afb454a76cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:25.872509 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa32399-b628-4f44-962f-4afb454a76cd admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:25.885317 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1348/2667] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:04:25.889655 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d36c6-67a1-4332-918b-d5ab3d8a16f5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:25.899493 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d36c6-67a1-4332-918b-d5ab3d8a16f5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:25.904174 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71556257-0c33-48c3-b03b-d4b46a6e9ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:25.904778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71556257-0c33-48c3-b03b-d4b46a6e9ecc admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:25.905083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71556257-0c33-48c3-b03b-d4b46a6e9ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:25.905351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71556257-0c33-48c3-b03b-d4b46a6e9ecc admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:25.905987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71556257-0c33-48c3-b03b-d4b46a6e9ecc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:25.907918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71556257-0c33-48c3-b03b-d4b46a6e9ecc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:26.036451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1320/2668] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:04:25 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:04:26.038166 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa32399-b628-4f44-962f-4afb454a76cd admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:26.049435 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa32399-b628-4f44-962f-4afb454a76cd admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:26.105546 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e857c7a-0a39-4d53-8859-8211de8c3394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:26.106149 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e857c7a-0a39-4d53-8859-8211de8c3394 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:26.106430 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e857c7a-0a39-4d53-8859-8211de8c3394 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:26.106704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e857c7a-0a39-4d53-8859-8211de8c3394 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:26.107394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e857c7a-0a39-4d53-8859-8211de8c3394 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:26.109940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e857c7a-0a39-4d53-8859-8211de8c3394 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:26.203250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1349/2669] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:04:26 2022] POST /identity/v3/users => generated 291 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:04:26.207566 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71556257-0c33-48c3-b03b-d4b46a6e9ecc admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:26.218515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71556257-0c33-48c3-b03b-d4b46a6e9ecc admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:26.225782 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b43423-541e-46c3-9f32-6fd13bbe0330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:26.226387 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b43423-541e-46c3-9f32-6fd13bbe0330 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:26.226667 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b43423-541e-46c3-9f32-6fd13bbe0330 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:26.226937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b43423-541e-46c3-9f32-6fd13bbe0330 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:26.227624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b43423-541e-46c3-9f32-6fd13bbe0330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:26.229256 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b43423-541e-46c3-9f32-6fd13bbe0330 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:26.241161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1321/2670] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:04:26.245305 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e857c7a-0a39-4d53-8859-8211de8c3394 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:26.255426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e857c7a-0a39-4d53-8859-8211de8c3394 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:26.260060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b200de-a590-4fd8-a172-fb3b104d21e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:26.260642 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b200de-a590-4fd8-a172-fb3b104d21e5 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:26.260944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b200de-a590-4fd8-a172-fb3b104d21e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:26.261217 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b200de-a590-4fd8-a172-fb3b104d21e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:26.261856 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b200de-a590-4fd8-a172-fb3b104d21e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:26.263570 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b200de-a590-4fd8-a172-fb3b104d21e5 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:26.274235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1350/2671] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:04:26.278272 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b43423-541e-46c3-9f32-6fd13bbe0330 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:26.288552 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b43423-541e-46c3-9f32-6fd13bbe0330 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:26.293076 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2432e9d-35fd-4941-9121-3ca21bc9ed59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:26.293704 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2432e9d-35fd-4941-9121-3ca21bc9ed59 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:26.293975 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2432e9d-35fd-4941-9121-3ca21bc9ed59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:26.294242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2432e9d-35fd-4941-9121-3ca21bc9ed59 admin admin] PATH_INFO: `/v3/projects/38d883b22d414ba3bc97041636f0058c/users/200201d6417d4f1e8ebd4f1652e82ad7/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:26.308823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2432e9d-35fd-4941-9121-3ca21bc9ed59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d883b22d414ba3bc97041636f0058c, user_id=200201d6417d4f1e8ebd4f1652e82ad7, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:26.311517 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2432e9d-35fd-4941-9121-3ca21bc9ed59 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:26.409544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1322/2672] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:04:26 2022] PUT /identity/v3/projects/38d883b22d414ba3bc97041636f0058c/users/200201d6417d4f1e8ebd4f1652e82ad7/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:26.419776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137867d0-b4c3-4d01-a5db-67ef9d4f6baa None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:26.423291 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137867d0-b4c3-4d01-a5db-67ef9d4f6baa None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:26.423291 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137867d0-b4c3-4d01-a5db-67ef9d4f6baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:26.441961 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:26.449280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-137867d0-b4c3-4d01-a5db-67ef9d4f6baa None None] MFA Rules not processed for user `200201d6417d4f1e8ebd4f1652e82ad7`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:26.450538 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-137867d0-b4c3-4d01-a5db-67ef9d4f6baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:26.513388 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1351/2673] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:26.518393 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2432e9d-35fd-4941-9121-3ca21bc9ed59 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:26.579875 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2432e9d-35fd-4941-9121-3ca21bc9ed59 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:26.584508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f072323-c82f-47b3-b14b-3c750bd7ce85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:26.585112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f072323-c82f-47b3-b14b-3c750bd7ce85 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:26.585387 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f072323-c82f-47b3-b14b-3c750bd7ce85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:26.585658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f072323-c82f-47b3-b14b-3c750bd7ce85 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:26.586284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f072323-c82f-47b3-b14b-3c750bd7ce85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:26.588233 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f072323-c82f-47b3-b14b-3c750bd7ce85 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:26.667329 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1323/2674] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:04:26 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:04:26.671395 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137867d0-b4c3-4d01-a5db-67ef9d4f6baa None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:26.682374 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137867d0-b4c3-4d01-a5db-67ef9d4f6baa None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:26.738495 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235acd28-01ac-4938-99c0-951a4421d155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:26.739088 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235acd28-01ac-4938-99c0-951a4421d155 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:26.739393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235acd28-01ac-4938-99c0-951a4421d155 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:26.739664 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235acd28-01ac-4938-99c0-951a4421d155 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:26.740295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235acd28-01ac-4938-99c0-951a4421d155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:26.742207 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235acd28-01ac-4938-99c0-951a4421d155 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:26.809567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1352/2675] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:04:26 2022] POST /identity/v3/users => generated 298 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:04:26.813759 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f072323-c82f-47b3-b14b-3c750bd7ce85 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:26.824229 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f072323-c82f-47b3-b14b-3c750bd7ce85 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:26.830057 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98ae57-7fa5-47a8-9e99-1b3d93303ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:26.830652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98ae57-7fa5-47a8-9e99-1b3d93303ffb admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:26.830930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98ae57-7fa5-47a8-9e99-1b3d93303ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:26.831226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98ae57-7fa5-47a8-9e99-1b3d93303ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:26.831866 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98ae57-7fa5-47a8-9e99-1b3d93303ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:26.833460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98ae57-7fa5-47a8-9e99-1b3d93303ffb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:26.844502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1324/2676] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:04:26.848835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235acd28-01ac-4938-99c0-951a4421d155 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:26.858882 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235acd28-01ac-4938-99c0-951a4421d155 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:26.863590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb641e2-5716-4d06-89e4-5f37df21b6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:26.864228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb641e2-5716-4d06-89e4-5f37df21b6ba admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:26.864512 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb641e2-5716-4d06-89e4-5f37df21b6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:26.864784 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb641e2-5716-4d06-89e4-5f37df21b6ba admin admin] PATH_INFO: `/v3/projects/d2803c04a31b4ef6bd64f3c591ac25d6/users/9b721551ec54478b9f59ba447c7f5fea/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:26.879970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb641e2-5716-4d06-89e4-5f37df21b6ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2803c04a31b4ef6bd64f3c591ac25d6, user_id=9b721551ec54478b9f59ba447c7f5fea, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:26.882618 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb641e2-5716-4d06-89e4-5f37df21b6ba admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:26.958415 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1353/2677] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:04:26 2022] PUT /identity/v3/projects/d2803c04a31b4ef6bd64f3c591ac25d6/users/9b721551ec54478b9f59ba447c7f5fea/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:26.962469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98ae57-7fa5-47a8-9e99-1b3d93303ffb admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:26.972794 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98ae57-7fa5-47a8-9e99-1b3d93303ffb admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:27.006454 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:27.031730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0303acbc-9e0c-4323-9050-3b3592483ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:27.032337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303acbc-9e0c-4323-9050-3b3592483ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:27.032611 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303acbc-9e0c-4323-9050-3b3592483ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:27.032877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303acbc-9e0c-4323-9050-3b3592483ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:27.033518 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0303acbc-9e0c-4323-9050-3b3592483ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:27.035152 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0303acbc-9e0c-4323-9050-3b3592483ff4 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:27.045821 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1325/2678] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:04:27.049630 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb641e2-5716-4d06-89e4-5f37df21b6ba admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:27.060401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb641e2-5716-4d06-89e4-5f37df21b6ba admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:27.067665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cca66-3dfb-4191-bdf1-fca62bb82d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:27.068322 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cca66-3dfb-4191-bdf1-fca62bb82d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:27.068598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cca66-3dfb-4191-bdf1-fca62bb82d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:27.068866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cca66-3dfb-4191-bdf1-fca62bb82d1d admin admin] PATH_INFO: `/v3/projects/d2803c04a31b4ef6bd64f3c591ac25d6/users/9b721551ec54478b9f59ba447c7f5fea/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:27.070928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46cca66-3dfb-4191-bdf1-fca62bb82d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2803c04a31b4ef6bd64f3c591ac25d6, user_id=9b721551ec54478b9f59ba447c7f5fea, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:27.073737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46cca66-3dfb-4191-bdf1-fca62bb82d1d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:27.085828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1354/2679] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:04:27 2022] PUT /identity/v3/projects/d2803c04a31b4ef6bd64f3c591ac25d6/users/9b721551ec54478b9f59ba447c7f5fea/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:27.097305 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a331f4-c3b2-4f47-977b-85c0f582dfd1 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:27.097863 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a331f4-c3b2-4f47-977b-85c0f582dfd1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:27.098122 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a331f4-c3b2-4f47-977b-85c0f582dfd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:27.125673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-56a331f4-c3b2-4f47-977b-85c0f582dfd1 None None] MFA Rules not processed for user `9b721551ec54478b9f59ba447c7f5fea`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:27.126885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-56a331f4-c3b2-4f47-977b-85c0f582dfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:27.196339 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1326/2680] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:27.207562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cca66-3dfb-4191-bdf1-fca62bb82d1d admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:27.209246 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b46cca66-3dfb-4191-bdf1-fca62bb82d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:27.267859 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cca66-3dfb-4191-bdf1-fca62bb82d1d admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:27.271041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd55df-16d1-402c-b0a8-fa3fcdd9e6e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:27.271667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd55df-16d1-402c-b0a8-fa3fcdd9e6e3 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:27.271945 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd55df-16d1-402c-b0a8-fa3fcdd9e6e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:27.272217 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd55df-16d1-402c-b0a8-fa3fcdd9e6e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:27.279134 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:27.281406 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd55df-16d1-402c-b0a8-fa3fcdd9e6e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:27.282861 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd55df-16d1-402c-b0a8-fa3fcdd9e6e3 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:27.292336 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1355/2681] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:27.393358 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a331f4-c3b2-4f47-977b-85c0f582dfd1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:27.402869 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a331f4-c3b2-4f47-977b-85c0f582dfd1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:27.406413 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158d00a-0ff5-4854-b306-454182bcf0f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:27.406998 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158d00a-0ff5-4854-b306-454182bcf0f3 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:27.407309 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158d00a-0ff5-4854-b306-454182bcf0f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:27.407578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158d00a-0ff5-4854-b306-454182bcf0f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:27.461898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158d00a-0ff5-4854-b306-454182bcf0f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:27.463360 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158d00a-0ff5-4854-b306-454182bcf0f3 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:27.472144 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1327/2682] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:30.580869 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd55df-16d1-402c-b0a8-fa3fcdd9e6e3 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:30.655334 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd55df-16d1-402c-b0a8-fa3fcdd9e6e3 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:30.660314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6bc23-b50b-4ca4-bf49-0c91b7a21de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:30.660958 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6bc23-b50b-4ca4-bf49-0c91b7a21de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:30.661236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6bc23-b50b-4ca4-bf49-0c91b7a21de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:30.661538 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6bc23-b50b-4ca4-bf49-0c91b7a21de2 admin admin] PATH_INFO: `/v3/users/200201d6417d4f1e8ebd4f1652e82ad7` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:30.662677 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6bc23-b50b-4ca4-bf49-0c91b7a21de2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=200201d6417d4f1e8ebd4f1652e82ad7)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:30.664707 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6bc23-b50b-4ca4-bf49-0c91b7a21de2 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:30.915063 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1bc6bc23-b50b-4ca4-bf49-0c91b7a21de2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '200201d6417d4f1e8ebd4f1652e82ad7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:30.923005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1bc6bc23-b50b-4ca4-bf49-0c91b7a21de2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '200201d6417d4f1e8ebd4f1652e82ad7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:30.935520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1bc6bc23-b50b-4ca4-bf49-0c91b7a21de2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '200201d6417d4f1e8ebd4f1652e82ad7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:30.936249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1bc6bc23-b50b-4ca4-bf49-0c91b7a21de2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '200201d6417d4f1e8ebd4f1652e82ad7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:30.936863 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-1bc6bc23-b50b-4ca4-bf49-0c91b7a21de2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '200201d6417d4f1e8ebd4f1652e82ad7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:31.095953 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1356/2683] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:04:30 2022] DELETE /identity/v3/users/200201d6417d4f1e8ebd4f1652e82ad7 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:31.100864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9069e32-99ee-441f-9eba-e474de323856 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:31.101313 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9069e32-99ee-441f-9eba-e474de323856 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:31.101441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9069e32-99ee-441f-9eba-e474de323856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:31.120368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-b9069e32-99ee-441f-9eba-e474de323856 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:31.121567 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b9069e32-99ee-441f-9eba-e474de323856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:31.171234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:31.181967 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1328/2684] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:31.191681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6bc23-b50b-4ca4-bf49-0c91b7a21de2 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:31.193152 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-1bc6bc23-b50b-4ca4-bf49-0c91b7a21de2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:31.255935 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6bc23-b50b-4ca4-bf49-0c91b7a21de2 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:31.259519 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874a85f-5c3e-4eb2-b3a5-fc315cdd0ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:31.260121 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874a85f-5c3e-4eb2-b3a5-fc315cdd0ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:31.263884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874a85f-5c3e-4eb2-b3a5-fc315cdd0ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:31.263884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874a85f-5c3e-4eb2-b3a5-fc315cdd0ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:31.270190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874a85f-5c3e-4eb2-b3a5-fc315cdd0ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:31.271697 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874a85f-5c3e-4eb2-b3a5-fc315cdd0ed4 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:31.281685 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1357/2685] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:31.937255 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9069e32-99ee-441f-9eba-e474de323856 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:31.938711 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b9069e32-99ee-441f-9eba-e474de323856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:31.950915 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9069e32-99ee-441f-9eba-e474de323856 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:31.955546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c896ef9-472e-4303-ab95-97d160414383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:31.956241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c896ef9-472e-4303-ab95-97d160414383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:31.956523 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c896ef9-472e-4303-ab95-97d160414383 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:31.956793 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c896ef9-472e-4303-ab95-97d160414383 admin admin] PATH_INFO: `/v3/projects/38d883b22d414ba3bc97041636f0058c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:31.957908 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c896ef9-472e-4303-ab95-97d160414383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38d883b22d414ba3bc97041636f0058c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:31.959884 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c896ef9-472e-4303-ab95-97d160414383 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:32.063291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8c896ef9-472e-4303-ab95-97d160414383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d883b22d414ba3bc97041636f0058c', 'request_id': 'req-8c896ef9-472e-4303-ab95-97d160414383'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:32.069457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8c896ef9-472e-4303-ab95-97d160414383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d883b22d414ba3bc97041636f0058c', 'request_id': 'req-8c896ef9-472e-4303-ab95-97d160414383'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:32.075208 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8c896ef9-472e-4303-ab95-97d160414383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d883b22d414ba3bc97041636f0058c', 'request_id': 'req-8c896ef9-472e-4303-ab95-97d160414383'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:32.174993 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8c896ef9-472e-4303-ab95-97d160414383 admin admin] The token cache is being invalidate because project 38d883b22d414ba3bc97041636f0058c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:04:32.176776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8c896ef9-472e-4303-ab95-97d160414383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:32.176776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8c896ef9-472e-4303-ab95-97d160414383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:32.177418 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1329/2686] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:04:31 2022] DELETE /identity/v3/projects/38d883b22d414ba3bc97041636f0058c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:32.181633 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874a85f-5c3e-4eb2-b3a5-fc315cdd0ed4 service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:32.191528 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874a85f-5c3e-4eb2-b3a5-fc315cdd0ed4 service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:32.203797 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:32.253048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3ce55-dc4c-4581-92d6-21a05670232e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:32.253771 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3ce55-dc4c-4581-92d6-21a05670232e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:32.254252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3ce55-dc4c-4581-92d6-21a05670232e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:32.254688 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3ce55-dc4c-4581-92d6-21a05670232e admin admin] PATH_INFO: `/v3/users/9b721551ec54478b9f59ba447c7f5fea` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:32.256230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3ce55-dc4c-4581-92d6-21a05670232e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b721551ec54478b9f59ba447c7f5fea)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:32.258168 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3ce55-dc4c-4581-92d6-21a05670232e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:32.399459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7fd3ce55-dc4c-4581-92d6-21a05670232e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b721551ec54478b9f59ba447c7f5fea'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:32.406765 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7fd3ce55-dc4c-4581-92d6-21a05670232e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b721551ec54478b9f59ba447c7f5fea'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:32.418542 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7fd3ce55-dc4c-4581-92d6-21a05670232e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b721551ec54478b9f59ba447c7f5fea'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:32.419223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7fd3ce55-dc4c-4581-92d6-21a05670232e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b721551ec54478b9f59ba447c7f5fea'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:32.419824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7fd3ce55-dc4c-4581-92d6-21a05670232e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b721551ec54478b9f59ba447c7f5fea'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:32.465643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1358/2687] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:04:32 2022] DELETE /identity/v3/users/9b721551ec54478b9f59ba447c7f5fea => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:32.838845 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c896ef9-472e-4303-ab95-97d160414383 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:32.840441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8c896ef9-472e-4303-ab95-97d160414383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:32.918247 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c896ef9-472e-4303-ab95-97d160414383 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:32.975169 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe9c7e-beda-480c-9e4f-54faa3f42f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:32.975812 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe9c7e-beda-480c-9e4f-54faa3f42f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:32.976089 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe9c7e-beda-480c-9e4f-54faa3f42f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:32.976399 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe9c7e-beda-480c-9e4f-54faa3f42f70 admin admin] PATH_INFO: `/v3/projects/d2803c04a31b4ef6bd64f3c591ac25d6` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:32.977513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe9c7e-beda-480c-9e4f-54faa3f42f70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2803c04a31b4ef6bd64f3c591ac25d6)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:32.979477 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe9c7e-beda-480c-9e4f-54faa3f42f70 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:33.005942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:33.072898 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0fbe9c7e-beda-480c-9e4f-54faa3f42f70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2803c04a31b4ef6bd64f3c591ac25d6', 'request_id': 'req-0fbe9c7e-beda-480c-9e4f-54faa3f42f70'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:33.080128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0fbe9c7e-beda-480c-9e4f-54faa3f42f70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2803c04a31b4ef6bd64f3c591ac25d6', 'request_id': 'req-0fbe9c7e-beda-480c-9e4f-54faa3f42f70'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:33.087333 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0fbe9c7e-beda-480c-9e4f-54faa3f42f70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2803c04a31b4ef6bd64f3c591ac25d6', 'request_id': 'req-0fbe9c7e-beda-480c-9e4f-54faa3f42f70'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:33.151422 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0fbe9c7e-beda-480c-9e4f-54faa3f42f70 admin admin] The token cache is being invalidate because project d2803c04a31b4ef6bd64f3c591ac25d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:04:33.151713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0fbe9c7e-beda-480c-9e4f-54faa3f42f70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:33.152336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-0fbe9c7e-beda-480c-9e4f-54faa3f42f70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:33.153807 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1330/2688] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:04:32 2022] DELETE /identity/v3/projects/d2803c04a31b4ef6bd64f3c591ac25d6 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:33.158795 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afe4ff-a3e6-4266-b6db-301d9297f180 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:33.159114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afe4ff-a3e6-4266-b6db-301d9297f180 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:33.159426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afe4ff-a3e6-4266-b6db-301d9297f180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:33.218330 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-83afe4ff-a3e6-4266-b6db-301d9297f180 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:33.219507 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-83afe4ff-a3e6-4266-b6db-301d9297f180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:33.278775 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1359/2689] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:33.286505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d7bd5-45b4-4fff-ab9f-d83d50dfb744 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:33.286957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d7bd5-45b4-4fff-ab9f-d83d50dfb744 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:33.287082 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d7bd5-45b4-4fff-ab9f-d83d50dfb744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:33.306739 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-436d7bd5-45b4-4fff-ab9f-d83d50dfb744 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:33.307966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-436d7bd5-45b4-4fff-ab9f-d83d50dfb744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:33.316351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1331/2690] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:33.320657 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afe4ff-a3e6-4266-b6db-301d9297f180 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:33.330085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afe4ff-a3e6-4266-b6db-301d9297f180 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:33.334581 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ffe5c-d0cb-4ea5-90ff-6fbb2cf137fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:33.335210 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ffe5c-d0cb-4ea5-90ff-6fbb2cf137fa admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:33.335489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ffe5c-d0cb-4ea5-90ff-6fbb2cf137fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:33.335761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ffe5c-d0cb-4ea5-90ff-6fbb2cf137fa admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:33.336400 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ffe5c-d0cb-4ea5-90ff-6fbb2cf137fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:04:33.336697 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ffe5c-d0cb-4ea5-90ff-6fbb2cf137fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:33.338277 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ffe5c-d0cb-4ea5-90ff-6fbb2cf137fa admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:33.339781 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:33.350284 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1360/2691] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:04:33.354596 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d7bd5-45b4-4fff-ab9f-d83d50dfb744 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:33.364404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d7bd5-45b4-4fff-ab9f-d83d50dfb744 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:33.369091 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fa9ce-c4c5-48ea-8d5f-380add83fecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:33.369684 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fa9ce-c4c5-48ea-8d5f-380add83fecc admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:33.371081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fa9ce-c4c5-48ea-8d5f-380add83fecc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:33.371081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042fa9ce-c4c5-48ea-8d5f-380add83fecc admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:33.371081 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042fa9ce-c4c5-48ea-8d5f-380add83fecc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:33.372857 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042fa9ce-c4c5-48ea-8d5f-380add83fecc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:33.466959 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1332/2692] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:04:33 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:04:33.471073 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ffe5c-d0cb-4ea5-90ff-6fbb2cf137fa admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:33.481147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ffe5c-d0cb-4ea5-90ff-6fbb2cf137fa admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:33.535591 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fa0dc-c3cd-4d9d-85be-afc71e372685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:33.536185 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fa0dc-c3cd-4d9d-85be-afc71e372685 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:33.536465 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fa0dc-c3cd-4d9d-85be-afc71e372685 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:33.536734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fa0dc-c3cd-4d9d-85be-afc71e372685 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:33.537354 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fa0dc-c3cd-4d9d-85be-afc71e372685 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:33.539294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fa0dc-c3cd-4d9d-85be-afc71e372685 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:33.615167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1361/2693] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:04:33 2022] POST /identity/v3/users => generated 291 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:04:33.624442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fa9ce-c4c5-48ea-8d5f-380add83fecc admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:33.634823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042fa9ce-c4c5-48ea-8d5f-380add83fecc admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:33.639488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb03d4-f883-48cb-a7af-06bfff1fac7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:33.640077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb03d4-f883-48cb-a7af-06bfff1fac7a admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:33.640353 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb03d4-f883-48cb-a7af-06bfff1fac7a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:33.640626 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb03d4-f883-48cb-a7af-06bfff1fac7a admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:33.641263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb03d4-f883-48cb-a7af-06bfff1fac7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:33.642917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb03d4-f883-48cb-a7af-06bfff1fac7a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:33.652713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1333/2694] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:04:33.657208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fa0dc-c3cd-4d9d-85be-afc71e372685 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:33.667613 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fa0dc-c3cd-4d9d-85be-afc71e372685 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:33.672964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c7939-d3a8-40f6-a615-f3954a3d3dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:33.673550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c7939-d3a8-40f6-a615-f3954a3d3dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:33.673829 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c7939-d3a8-40f6-a615-f3954a3d3dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:33.674130 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c7939-d3a8-40f6-a615-f3954a3d3dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:33.674765 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c7939-d3a8-40f6-a615-f3954a3d3dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:33.676703 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c7939-d3a8-40f6-a615-f3954a3d3dc5 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:33.687290 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1362/2695] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:04:33.691368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb03d4-f883-48cb-a7af-06bfff1fac7a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:33.701177 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb03d4-f883-48cb-a7af-06bfff1fac7a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:33.705877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe81faf-d544-4165-a4ea-6928b07dd2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:33.706517 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe81faf-d544-4165-a4ea-6928b07dd2fb admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:33.706790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe81faf-d544-4165-a4ea-6928b07dd2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:33.707056 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe81faf-d544-4165-a4ea-6928b07dd2fb admin admin] PATH_INFO: `/v3/projects/40884a6311604b5480d4589ea16dcb47/users/0169c25bac78427ea9ef0548de72161e/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:33.722234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe81faf-d544-4165-a4ea-6928b07dd2fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=40884a6311604b5480d4589ea16dcb47, user_id=0169c25bac78427ea9ef0548de72161e, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:33.725099 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe81faf-d544-4165-a4ea-6928b07dd2fb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:33.803551 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1334/2696] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:04:33 2022] PUT /identity/v3/projects/40884a6311604b5480d4589ea16dcb47/users/0169c25bac78427ea9ef0548de72161e/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:33.815374 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad24df-9274-4026-855f-013c949a7114 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:33.815660 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad24df-9274-4026-855f-013c949a7114 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:33.815934 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad24df-9274-4026-855f-013c949a7114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:33.843554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-f3ad24df-9274-4026-855f-013c949a7114 None None] MFA Rules not processed for user `0169c25bac78427ea9ef0548de72161e`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:33.844759 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-f3ad24df-9274-4026-855f-013c949a7114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:33.907569 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1363/2697] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:33.912110 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe81faf-d544-4165-a4ea-6928b07dd2fb admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:33.973496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe81faf-d544-4165-a4ea-6928b07dd2fb admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:33.978139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77c5d8-ad4b-47cb-9fb4-1827453d2c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:33.978732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77c5d8-ad4b-47cb-9fb4-1827453d2c68 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:33.979820 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77c5d8-ad4b-47cb-9fb4-1827453d2c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:33.979820 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77c5d8-ad4b-47cb-9fb4-1827453d2c68 admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:33.980003 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77c5d8-ad4b-47cb-9fb4-1827453d2c68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:33.981780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77c5d8-ad4b-47cb-9fb4-1827453d2c68 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:34.050068 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1335/2698] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:04:33 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:04:34.053993 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad24df-9274-4026-855f-013c949a7114 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:34.064658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad24df-9274-4026-855f-013c949a7114 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:34.120478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2a59a-7848-4e46-b386-e595fd738d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:34.121082 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2a59a-7848-4e46-b386-e595fd738d58 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:34.121359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2a59a-7848-4e46-b386-e595fd738d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:34.121630 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2a59a-7848-4e46-b386-e595fd738d58 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:34.122272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2a59a-7848-4e46-b386-e595fd738d58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:34.124262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2a59a-7848-4e46-b386-e595fd738d58 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:34.133464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:34.210016 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1364/2699] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:04:34 2022] POST /identity/v3/users => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:04:34.214075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77c5d8-ad4b-47cb-9fb4-1827453d2c68 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:34.231334 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77c5d8-ad4b-47cb-9fb4-1827453d2c68 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:34.231334 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda9174-ab15-4c98-821b-85117efe7597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:34.231795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda9174-ab15-4c98-821b-85117efe7597 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:34.232142 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda9174-ab15-4c98-821b-85117efe7597 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:34.232421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda9174-ab15-4c98-821b-85117efe7597 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:34.233075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda9174-ab15-4c98-821b-85117efe7597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:34.234736 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda9174-ab15-4c98-821b-85117efe7597 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:34.249052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1336/2700] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:04:34.253659 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2a59a-7848-4e46-b386-e595fd738d58 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:34.267820 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2a59a-7848-4e46-b386-e595fd738d58 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:34.273002 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875faee-7fbc-4e70-81fc-18908d32d912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:34.273792 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875faee-7fbc-4e70-81fc-18908d32d912 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:34.274132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875faee-7fbc-4e70-81fc-18908d32d912 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:34.274470 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875faee-7fbc-4e70-81fc-18908d32d912 admin admin] PATH_INFO: `/v3/projects/197867b40a3f44cf8b80697407a3dca7/users/0673b08d0f204adda08663b44f9f46c8/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:34.290413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875faee-7fbc-4e70-81fc-18908d32d912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=197867b40a3f44cf8b80697407a3dca7, user_id=0673b08d0f204adda08663b44f9f46c8, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:34.293188 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875faee-7fbc-4e70-81fc-18908d32d912 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:34.402916 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1365/2701] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:04:34 2022] PUT /identity/v3/projects/197867b40a3f44cf8b80697407a3dca7/users/0673b08d0f204adda08663b44f9f46c8/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:34.406988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda9174-ab15-4c98-821b-85117efe7597 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:34.416966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda9174-ab15-4c98-821b-85117efe7597 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:34.429241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:34.472967 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678042f-fe7c-44d6-8230-4bc0ae9a31eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:34.473556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678042f-fe7c-44d6-8230-4bc0ae9a31eb admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:34.473832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678042f-fe7c-44d6-8230-4bc0ae9a31eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:34.474100 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678042f-fe7c-44d6-8230-4bc0ae9a31eb admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:34.474728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678042f-fe7c-44d6-8230-4bc0ae9a31eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:34.476337 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678042f-fe7c-44d6-8230-4bc0ae9a31eb admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:34.486404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1337/2702] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:04:34.490492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875faee-7fbc-4e70-81fc-18908d32d912 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:34.501064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875faee-7fbc-4e70-81fc-18908d32d912 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:34.505589 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481f447-4c93-4945-9cb1-6bdceea9da47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:34.506225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481f447-4c93-4945-9cb1-6bdceea9da47 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:34.506498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481f447-4c93-4945-9cb1-6bdceea9da47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:34.506766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481f447-4c93-4945-9cb1-6bdceea9da47 admin admin] PATH_INFO: `/v3/projects/197867b40a3f44cf8b80697407a3dca7/users/0673b08d0f204adda08663b44f9f46c8/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:34.508694 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7481f447-4c93-4945-9cb1-6bdceea9da47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=197867b40a3f44cf8b80697407a3dca7, user_id=0673b08d0f204adda08663b44f9f46c8, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:34.511421 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7481f447-4c93-4945-9cb1-6bdceea9da47 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:34.522042 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1366/2703] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:04:34 2022] PUT /identity/v3/projects/197867b40a3f44cf8b80697407a3dca7/users/0673b08d0f204adda08663b44f9f46c8/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:34.531858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d385e-d3e5-4552-b3ea-72f82be2728f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:34.532144 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d385e-d3e5-4552-b3ea-72f82be2728f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:34.532419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d385e-d3e5-4552-b3ea-72f82be2728f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:34.560699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-fa4d385e-d3e5-4552-b3ea-72f82be2728f None None] MFA Rules not processed for user `0673b08d0f204adda08663b44f9f46c8`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:34.562379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fa4d385e-d3e5-4552-b3ea-72f82be2728f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:34.628395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1338/2704] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:04:34.640573 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481f447-4c93-4945-9cb1-6bdceea9da47 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:34.642408 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7481f447-4c93-4945-9cb1-6bdceea9da47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:34.717160 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481f447-4c93-4945-9cb1-6bdceea9da47 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:34.721098 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a4cc3-26db-43c5-bbf1-f655251d403f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:34.721713 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a4cc3-26db-43c5-bbf1-f655251d403f service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:34.721989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a4cc3-26db-43c5-bbf1-f655251d403f service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:34.722502 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a4cc3-26db-43c5-bbf1-f655251d403f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:34.731337 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a4cc3-26db-43c5-bbf1-f655251d403f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:34.732745 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a4cc3-26db-43c5-bbf1-f655251d403f service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:34.742571 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1367/2705] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:34.891790 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d385e-d3e5-4552-b3ea-72f82be2728f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:34.902224 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d385e-d3e5-4552-b3ea-72f82be2728f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:34.905767 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5e8bc-79a0-4925-b13f-029f910c857e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:34.906364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5e8bc-79a0-4925-b13f-029f910c857e service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:34.906634 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5e8bc-79a0-4925-b13f-029f910c857e service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:34.906907 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5e8bc-79a0-4925-b13f-029f910c857e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:34.962485 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5e8bc-79a0-4925-b13f-029f910c857e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:34.963948 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5e8bc-79a0-4925-b13f-029f910c857e service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:34.972810 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1339/2706] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:35.459738 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a4cc3-26db-43c5-bbf1-f655251d403f service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:35.521373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a4cc3-26db-43c5-bbf1-f655251d403f service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:35.525807 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179b5fa-b098-427b-9236-2b9968036ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:35.526419 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179b5fa-b098-427b-9236-2b9968036ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:35.526697 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179b5fa-b098-427b-9236-2b9968036ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:35.526968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179b5fa-b098-427b-9236-2b9968036ff6 admin admin] PATH_INFO: `/v3/users/0169c25bac78427ea9ef0548de72161e` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:35.528104 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4179b5fa-b098-427b-9236-2b9968036ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0169c25bac78427ea9ef0548de72161e)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:35.530039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4179b5fa-b098-427b-9236-2b9968036ff6 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:35.684307 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:INSERT=4,keystone:SELECT=90,keystone:DELETE=6 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:35.687836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-4179b5fa-b098-427b-9236-2b9968036ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0169c25bac78427ea9ef0548de72161e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:35.695428 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-4179b5fa-b098-427b-9236-2b9968036ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0169c25bac78427ea9ef0548de72161e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:35.708228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-4179b5fa-b098-427b-9236-2b9968036ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0169c25bac78427ea9ef0548de72161e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:35.708972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-4179b5fa-b098-427b-9236-2b9968036ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0169c25bac78427ea9ef0548de72161e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:35.709576 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-4179b5fa-b098-427b-9236-2b9968036ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0169c25bac78427ea9ef0548de72161e'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:35.791417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1368/2707] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:04:35 2022] DELETE /identity/v3/users/0169c25bac78427ea9ef0548de72161e => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:35.795881 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854879b-24e3-48cc-9b0f-ad9c239670f5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:35.796351 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854879b-24e3-48cc-9b0f-ad9c239670f5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:35.796475 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854879b-24e3-48cc-9b0f-ad9c239670f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:35.815730 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-8854879b-24e3-48cc-9b0f-ad9c239670f5 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:35.816950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8854879b-24e3-48cc-9b0f-ad9c239670f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:35.849713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=100 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:35.875364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1340/2708] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:35.884887 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179b5fa-b098-427b-9236-2b9968036ff6 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:35.886274 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-4179b5fa-b098-427b-9236-2b9968036ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:35.943507 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179b5fa-b098-427b-9236-2b9968036ff6 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:35.947258 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bab9ea-e994-43e3-8d4e-ff1868503370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:35.947846 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bab9ea-e994-43e3-8d4e-ff1868503370 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:35.948119 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bab9ea-e994-43e3-8d4e-ff1868503370 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:35.948386 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bab9ea-e994-43e3-8d4e-ff1868503370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:35.957761 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bab9ea-e994-43e3-8d4e-ff1868503370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:35.959281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bab9ea-e994-43e3-8d4e-ff1868503370 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:35.968734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1369/2709] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:36.416127 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854879b-24e3-48cc-9b0f-ad9c239670f5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:36.417592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8854879b-24e3-48cc-9b0f-ad9c239670f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:36.431160 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854879b-24e3-48cc-9b0f-ad9c239670f5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:36.436024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:36.437622 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:36.437745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:36.437974 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab admin admin] PATH_INFO: `/v3/projects/40884a6311604b5480d4589ea16dcb47` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:36.439243 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=40884a6311604b5480d4589ea16dcb47)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:36.441266 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:36.529409 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40884a6311604b5480d4589ea16dcb47', 'request_id': 'req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:36.538885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40884a6311604b5480d4589ea16dcb47', 'request_id': 'req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:36.546603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40884a6311604b5480d4589ea16dcb47', 'request_id': 'req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:36.615378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab admin admin] The token cache is being invalidate because project 40884a6311604b5480d4589ea16dcb47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:04:36.615659 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:36.616312 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:36.617816 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1341/2710] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:04:36 2022] DELETE /identity/v3/projects/40884a6311604b5480d4589ea16dcb47 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:36.622104 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bab9ea-e994-43e3-8d4e-ff1868503370 service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:36.631811 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bab9ea-e994-43e3-8d4e-ff1868503370 service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:36.688227 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368ea1b-9b7d-4961-a620-f204758bbf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:36.688787 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368ea1b-9b7d-4961-a620-f204758bbf61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:36.689069 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368ea1b-9b7d-4961-a620-f204758bbf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:36.689343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368ea1b-9b7d-4961-a620-f204758bbf61 admin admin] PATH_INFO: `/v3/users/0673b08d0f204adda08663b44f9f46c8` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:36.690459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368ea1b-9b7d-4961-a620-f204758bbf61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0673b08d0f204adda08663b44f9f46c8)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:36.692476 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368ea1b-9b7d-4961-a620-f204758bbf61 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:36.845030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-5368ea1b-9b7d-4961-a620-f204758bbf61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0673b08d0f204adda08663b44f9f46c8'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:36.852059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-5368ea1b-9b7d-4961-a620-f204758bbf61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0673b08d0f204adda08663b44f9f46c8'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:36.859431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:36.864778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-5368ea1b-9b7d-4961-a620-f204758bbf61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0673b08d0f204adda08663b44f9f46c8'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:36.866345 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-5368ea1b-9b7d-4961-a620-f204758bbf61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0673b08d0f204adda08663b44f9f46c8'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:36.866345 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-5368ea1b-9b7d-4961-a620-f204758bbf61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0673b08d0f204adda08663b44f9f46c8'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:36.962908 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1370/2711] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:04:36 2022] DELETE /identity/v3/users/0673b08d0f204adda08663b44f9f46c8 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:04:37.387434 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:37.388917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:37.402992 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5791f-a55f-44d9-af6c-6eaf2b40d4ab admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:37.459079 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383aa8a-d1a6-41e8-99e7-fddc804ae079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:37.459734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383aa8a-d1a6-41e8-99e7-fddc804ae079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:37.460037 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383aa8a-d1a6-41e8-99e7-fddc804ae079 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:37.460307 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383aa8a-d1a6-41e8-99e7-fddc804ae079 admin admin] PATH_INFO: `/v3/projects/197867b40a3f44cf8b80697407a3dca7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:37.461461 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383aa8a-d1a6-41e8-99e7-fddc804ae079 admin admin] RBAC: Authorizing `identity:delete_project(project_id=197867b40a3f44cf8b80697407a3dca7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:37.463672 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383aa8a-d1a6-41e8-99e7-fddc804ae079 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:37.568547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7383aa8a-d1a6-41e8-99e7-fddc804ae079 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '197867b40a3f44cf8b80697407a3dca7', 'request_id': 'req-7383aa8a-d1a6-41e8-99e7-fddc804ae079'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:37.574494 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7383aa8a-d1a6-41e8-99e7-fddc804ae079 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '197867b40a3f44cf8b80697407a3dca7', 'request_id': 'req-7383aa8a-d1a6-41e8-99e7-fddc804ae079'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:37.580544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7383aa8a-d1a6-41e8-99e7-fddc804ae079 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '197867b40a3f44cf8b80697407a3dca7', 'request_id': 'req-7383aa8a-d1a6-41e8-99e7-fddc804ae079'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:37.582142 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:37.665213 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7383aa8a-d1a6-41e8-99e7-fddc804ae079 admin admin] The token cache is being invalidate because project 197867b40a3f44cf8b80697407a3dca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:04:37.665529 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7383aa8a-d1a6-41e8-99e7-fddc804ae079 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:37.666149 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7383aa8a-d1a6-41e8-99e7-fddc804ae079 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:37.667931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1342/2712] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:04:37 2022] DELETE /identity/v3/projects/197867b40a3f44cf8b80697407a3dca7 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:37.673755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717c2c6-d5ac-43ba-9431-2faae2673630 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:37.673755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717c2c6-d5ac-43ba-9431-2faae2673630 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:37.673755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717c2c6-d5ac-43ba-9431-2faae2673630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:37.694827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-0717c2c6-d5ac-43ba-9431-2faae2673630 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:37.695560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0717c2c6-d5ac-43ba-9431-2faae2673630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:37.763732 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1371/2713] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:04:37.771632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119b15c-5177-4c09-a844-adcd7576e5cf None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:37.772666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119b15c-5177-4c09-a844-adcd7576e5cf None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:37.772950 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119b15c-5177-4c09-a844-adcd7576e5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:37.802969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-0119b15c-5177-4c09-a844-adcd7576e5cf None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:37.804286 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0119b15c-5177-4c09-a844-adcd7576e5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:37.816580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1343/2714] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:37.817762 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717c2c6-d5ac-43ba-9431-2faae2673630 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:37.828166 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717c2c6-d5ac-43ba-9431-2faae2673630 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:37.833630 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf42c81-d32a-43a8-8e4d-820f5c0a1a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:37.834232 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf42c81-d32a-43a8-8e4d-820f5c0a1a62 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:37.834511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf42c81-d32a-43a8-8e4d-820f5c0a1a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:37.834777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf42c81-d32a-43a8-8e4d-820f5c0a1a62 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:37.835729 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf42c81-d32a-43a8-8e4d-820f5c0a1a62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:04:37.836033 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf42c81-d32a-43a8-8e4d-820f5c0a1a62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:37.838986 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf42c81-d32a-43a8-8e4d-820f5c0a1a62 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:37.852137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1372/2715] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:04:37.856593 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119b15c-5177-4c09-a844-adcd7576e5cf None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:37.867808 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119b15c-5177-4c09-a844-adcd7576e5cf None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:37.872941 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880149f-888f-46cd-a4ed-13ed6095a2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:37.873544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880149f-888f-46cd-a4ed-13ed6095a2ec admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:37.874433 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880149f-888f-46cd-a4ed-13ed6095a2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:37.874701 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880149f-888f-46cd-a4ed-13ed6095a2ec admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:37.875377 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880149f-888f-46cd-a4ed-13ed6095a2ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:37.877316 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880149f-888f-46cd-a4ed-13ed6095a2ec admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:37.970551 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1344/2716] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:04:37 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:04:37.975006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf42c81-d32a-43a8-8e4d-820f5c0a1a62 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:37.985381 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf42c81-d32a-43a8-8e4d-820f5c0a1a62 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:38.046349 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039552c-427b-4847-9b37-7eea7c52fb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:38.046953 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039552c-427b-4847-9b37-7eea7c52fb3a admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:38.047259 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039552c-427b-4847-9b37-7eea7c52fb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:38.047530 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039552c-427b-4847-9b37-7eea7c52fb3a admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:38.048169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039552c-427b-4847-9b37-7eea7c52fb3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:38.050065 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039552c-427b-4847-9b37-7eea7c52fb3a admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:38.129182 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1373/2717] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:04:37 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:04:38.133240 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880149f-888f-46cd-a4ed-13ed6095a2ec admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:38.143753 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880149f-888f-46cd-a4ed-13ed6095a2ec admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:38.148475 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c3dbb-e081-4910-8945-7d926550863e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:38.149102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c3dbb-e081-4910-8945-7d926550863e admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:38.149388 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c3dbb-e081-4910-8945-7d926550863e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:38.149667 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c3dbb-e081-4910-8945-7d926550863e admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:38.150318 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c3dbb-e081-4910-8945-7d926550863e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:38.151974 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c3dbb-e081-4910-8945-7d926550863e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:38.165705 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1345/2718] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:04:38.166827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039552c-427b-4847-9b37-7eea7c52fb3a admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:38.177773 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039552c-427b-4847-9b37-7eea7c52fb3a admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:38.182459 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ba52f-c475-4425-8914-2cfa385fd948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:38.183049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ba52f-c475-4425-8914-2cfa385fd948 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:38.183364 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ba52f-c475-4425-8914-2cfa385fd948 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:38.183638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ba52f-c475-4425-8914-2cfa385fd948 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:38.184280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ba52f-c475-4425-8914-2cfa385fd948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:38.185980 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ba52f-c475-4425-8914-2cfa385fd948 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:38.196463 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1374/2719] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:04:38.200502 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c3dbb-e081-4910-8945-7d926550863e admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:38.210497 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c3dbb-e081-4910-8945-7d926550863e admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:38.215160 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7f972-2859-4ea3-955e-320451040f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:38.215797 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7f972-2859-4ea3-955e-320451040f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:38.216074 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7f972-2859-4ea3-955e-320451040f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:38.216345 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7f972-2859-4ea3-955e-320451040f2b admin admin] PATH_INFO: `/v3/projects/0aed44c0b05b4e0aac7acb77ac40ee22/users/ecfb22a4e5f04fce8fb65f39d34a962c/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:38.232936 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7f972-2859-4ea3-955e-320451040f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aed44c0b05b4e0aac7acb77ac40ee22, user_id=ecfb22a4e5f04fce8fb65f39d34a962c, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:38.236942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7f972-2859-4ea3-955e-320451040f2b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:38.330607 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1346/2720] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:04:38 2022] PUT /identity/v3/projects/0aed44c0b05b4e0aac7acb77ac40ee22/users/ecfb22a4e5f04fce8fb65f39d34a962c/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:38.342005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167edc7-b804-4fcc-ba3d-b422c96a628b None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:38.342317 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167edc7-b804-4fcc-ba3d-b422c96a628b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:38.343135 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167edc7-b804-4fcc-ba3d-b422c96a628b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:38.372319 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-9167edc7-b804-4fcc-ba3d-b422c96a628b None None] MFA Rules not processed for user `ecfb22a4e5f04fce8fb65f39d34a962c`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:38.373581 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9167edc7-b804-4fcc-ba3d-b422c96a628b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:38.413600 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:38.439444 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1375/2721] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:04:38.450300 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7f972-2859-4ea3-955e-320451040f2b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:38.452538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-25f7f972-2859-4ea3-955e-320451040f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:38.507994 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7f972-2859-4ea3-955e-320451040f2b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:38.511733 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344f619-7dd9-402f-a0fc-79c3f049c211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:38.512327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344f619-7dd9-402f-a0fc-79c3f049c211 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:38.512606 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344f619-7dd9-402f-a0fc-79c3f049c211 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:38.512878 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344f619-7dd9-402f-a0fc-79c3f049c211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:38.521425 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344f619-7dd9-402f-a0fc-79c3f049c211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:38.522847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344f619-7dd9-402f-a0fc-79c3f049c211 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:38.532031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1347/2722] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:41.370966 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e53fe6-5d4e-4045-a05b-d5d9d7332112 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:41.371513 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e53fe6-5d4e-4045-a05b-d5d9d7332112 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:41.371513 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e53fe6-5d4e-4045-a05b-d5d9d7332112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:41.391295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-29e53fe6-5d4e-4045-a05b-d5d9d7332112 None None] MFA Rules not processed for user `6a9784d49f8d45e38f096de3b2401d36`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:41.392495 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-29e53fe6-5d4e-4045-a05b-d5d9d7332112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:41.447117 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1376/2723] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:43.919533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344f619-7dd9-402f-a0fc-79c3f049c211 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:43.920951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-0344f619-7dd9-402f-a0fc-79c3f049c211 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:44.012422 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344f619-7dd9-402f-a0fc-79c3f049c211 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:44.018147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2b1b6-1d3f-4080-a3a5-6880610078f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4f11655cd51438e9652869f74ffa975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:44.018788 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2b1b6-1d3f-4080-a3a5-6880610078f0 service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:44.019066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2b1b6-1d3f-4080-a3a5-6880610078f0 service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:44.019363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2b1b6-1d3f-4080-a3a5-6880610078f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:44.021070 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bfd2b1b6-1d3f-4080-a3a5-6880610078f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:44.033409 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2b1b6-1d3f-4080-a3a5-6880610078f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:44.035327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2b1b6-1d3f-4080-a3a5-6880610078f0 service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:44.040527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:44.048912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1348/2724] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:44.051013 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e53fe6-5d4e-4045-a05b-d5d9d7332112 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:44.052429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-29e53fe6-5d4e-4045-a05b-d5d9d7332112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:44.067206 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e53fe6-5d4e-4045-a05b-d5d9d7332112 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:44.072294 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b636a-3738-4b80-88b1-ac12022dd590 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4f11655cd51438e9652869f74ffa975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:44.072885 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b636a-3738-4b80-88b1-ac12022dd590 service placement] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:44.073179 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b636a-3738-4b80-88b1-ac12022dd590 service placement] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:44.073445 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b636a-3738-4b80-88b1-ac12022dd590 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:44.082882 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b636a-3738-4b80-88b1-ac12022dd590 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:44.084630 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b636a-3738-4b80-88b1-ac12022dd590 service placement] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:44.095739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1377/2725] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:48.457010 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2b1b6-1d3f-4080-a3a5-6880610078f0 service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:48.517212 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2b1b6-1d3f-4080-a3a5-6880610078f0 service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:48.521666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731d68a-97ae-432b-99cd-6a6c4c95860f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:48.522271 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731d68a-97ae-432b-99cd-6a6c4c95860f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:48.522544 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731d68a-97ae-432b-99cd-6a6c4c95860f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:48.522811 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731d68a-97ae-432b-99cd-6a6c4c95860f admin admin] PATH_INFO: `/v3/users/ecfb22a4e5f04fce8fb65f39d34a962c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:48.523928 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731d68a-97ae-432b-99cd-6a6c4c95860f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecfb22a4e5f04fce8fb65f39d34a962c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:48.525859 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731d68a-97ae-432b-99cd-6a6c4c95860f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:48.679067 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d731d68a-97ae-432b-99cd-6a6c4c95860f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecfb22a4e5f04fce8fb65f39d34a962c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:48.686470 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d731d68a-97ae-432b-99cd-6a6c4c95860f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecfb22a4e5f04fce8fb65f39d34a962c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:48.687111 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d731d68a-97ae-432b-99cd-6a6c4c95860f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecfb22a4e5f04fce8fb65f39d34a962c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:48.739686 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d731d68a-97ae-432b-99cd-6a6c4c95860f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecfb22a4e5f04fce8fb65f39d34a962c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:48.752291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d731d68a-97ae-432b-99cd-6a6c4c95860f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecfb22a4e5f04fce8fb65f39d34a962c'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:48.758340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1349/2726] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:04:48 2022] DELETE /identity/v3/users/ecfb22a4e5f04fce8fb65f39d34a962c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:48.762962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dc0fc-b539-4560-99ad-9848cd8fab94 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:48.763478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dc0fc-b539-4560-99ad-9848cd8fab94 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:48.763594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dc0fc-b539-4560-99ad-9848cd8fab94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:48.782584 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-8f0dc0fc-b539-4560-99ad-9848cd8fab94 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:48.783800 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8f0dc0fc-b539-4560-99ad-9848cd8fab94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:48.842586 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1378/2727] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:04:48.852677 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731d68a-97ae-432b-99cd-6a6c4c95860f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:48.854071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d731d68a-97ae-432b-99cd-6a6c4c95860f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:48.914443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731d68a-97ae-432b-99cd-6a6c4c95860f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:48.918111 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198918b-70f9-4a0f-99f6-ba4359b6d1f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:48.918714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198918b-70f9-4a0f-99f6-ba4359b6d1f2 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:48.918985 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198918b-70f9-4a0f-99f6-ba4359b6d1f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:48.919284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198918b-70f9-4a0f-99f6-ba4359b6d1f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:48.928512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198918b-70f9-4a0f-99f6-ba4359b6d1f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:48.929974 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198918b-70f9-4a0f-99f6-ba4359b6d1f2 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:48.940172 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1350/2728] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:49.319333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0dc0fc-b539-4560-99ad-9848cd8fab94 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:49.320780 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8f0dc0fc-b539-4560-99ad-9848cd8fab94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:49.332896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0dc0fc-b539-4560-99ad-9848cd8fab94 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:49.337605 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff5a89-cf42-43ce-b993-62e73cae0915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:49.338234 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff5a89-cf42-43ce-b993-62e73cae0915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:49.338508 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff5a89-cf42-43ce-b993-62e73cae0915 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:49.338803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff5a89-cf42-43ce-b993-62e73cae0915 admin admin] PATH_INFO: `/v3/projects/0aed44c0b05b4e0aac7acb77ac40ee22` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:49.339969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff5a89-cf42-43ce-b993-62e73cae0915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aed44c0b05b4e0aac7acb77ac40ee22)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:49.341876 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff5a89-cf42-43ce-b993-62e73cae0915 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:49.465823 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f5ff5a89-cf42-43ce-b993-62e73cae0915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aed44c0b05b4e0aac7acb77ac40ee22', 'request_id': 'req-f5ff5a89-cf42-43ce-b993-62e73cae0915'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:49.471778 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f5ff5a89-cf42-43ce-b993-62e73cae0915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aed44c0b05b4e0aac7acb77ac40ee22', 'request_id': 'req-f5ff5a89-cf42-43ce-b993-62e73cae0915'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:49.546155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f5ff5a89-cf42-43ce-b993-62e73cae0915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aed44c0b05b4e0aac7acb77ac40ee22', 'request_id': 'req-f5ff5a89-cf42-43ce-b993-62e73cae0915'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:49.547350 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:49.557739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f5ff5a89-cf42-43ce-b993-62e73cae0915 admin admin] The token cache is being invalidate because project 0aed44c0b05b4e0aac7acb77ac40ee22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:04:49.558026 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f5ff5a89-cf42-43ce-b993-62e73cae0915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:49.558642 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f5ff5a89-cf42-43ce-b993-62e73cae0915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:04:49.560159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1379/2729] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:04:49 2022] DELETE /identity/v3/projects/0aed44c0b05b4e0aac7acb77ac40ee22 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:04:49.564945 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaff5b-484c-44ab-954c-e1d8f74b5510 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:49.565417 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaff5b-484c-44ab-954c-e1d8f74b5510 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:49.565515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaff5b-484c-44ab-954c-e1d8f74b5510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:49.584490 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-1afaff5b-484c-44ab-954c-e1d8f74b5510 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:49.585713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-1afaff5b-484c-44ab-954c-e1d8f74b5510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:49.602718 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:49.642806 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1351/2730] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:49.650751 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d606925-fd87-4648-b03b-da2d80835cbd None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:49.651033 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d606925-fd87-4648-b03b-da2d80835cbd None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:49.651338 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d606925-fd87-4648-b03b-da2d80835cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:49.671636 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-2d606925-fd87-4648-b03b-da2d80835cbd None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:49.672827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-2d606925-fd87-4648-b03b-da2d80835cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:49.681181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1380/2731] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:49.685539 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaff5b-484c-44ab-954c-e1d8f74b5510 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:49.695173 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaff5b-484c-44ab-954c-e1d8f74b5510 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:49.699973 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac6175-4e64-4a7f-b167-ec8e4d3a1efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:49.700937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac6175-4e64-4a7f-b167-ec8e4d3a1efc admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:49.701265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac6175-4e64-4a7f-b167-ec8e4d3a1efc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:49.702027 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac6175-4e64-4a7f-b167-ec8e4d3a1efc admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:49.702670 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac6175-4e64-4a7f-b167-ec8e4d3a1efc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:04:49.702973 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac6175-4e64-4a7f-b167-ec8e4d3a1efc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:49.704603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ac6175-4e64-4a7f-b167-ec8e4d3a1efc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:49.715811 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1352/2732] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:04:49.720717 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d606925-fd87-4648-b03b-da2d80835cbd None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:49.730140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d606925-fd87-4648-b03b-da2d80835cbd None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:49.738173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c7713-ae44-4b1a-933f-2db6e14548ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:49.738173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c7713-ae44-4b1a-933f-2db6e14548ac admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:49.738173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c7713-ae44-4b1a-933f-2db6e14548ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:49.738173 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c7713-ae44-4b1a-933f-2db6e14548ac admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:49.738568 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c7713-ae44-4b1a-933f-2db6e14548ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:49.738661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c7713-ae44-4b1a-933f-2db6e14548ac admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:49.815156 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1381/2733] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:04:49 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:04:49.819168 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac6175-4e64-4a7f-b167-ec8e4d3a1efc admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:49.829828 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac6175-4e64-4a7f-b167-ec8e4d3a1efc admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:49.885903 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99038d-f076-4001-9bb9-d42cba2850f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:49.886505 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99038d-f076-4001-9bb9-d42cba2850f7 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:49.886785 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99038d-f076-4001-9bb9-d42cba2850f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:49.887057 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99038d-f076-4001-9bb9-d42cba2850f7 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:49.887726 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99038d-f076-4001-9bb9-d42cba2850f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:49.890141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99038d-f076-4001-9bb9-d42cba2850f7 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:49.969520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1353/2734] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:04:49 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:04:49.973913 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c7713-ae44-4b1a-933f-2db6e14548ac admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:49.984124 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c7713-ae44-4b1a-933f-2db6e14548ac admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:49.988907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf832f0-1b52-4a92-b948-7da968ff4fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:49.989505 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf832f0-1b52-4a92-b948-7da968ff4fad admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:49.989783 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf832f0-1b52-4a92-b948-7da968ff4fad admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:49.990053 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf832f0-1b52-4a92-b948-7da968ff4fad admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:49.990695 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf832f0-1b52-4a92-b948-7da968ff4fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:49.992371 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf832f0-1b52-4a92-b948-7da968ff4fad admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:50.002919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1382/2735] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:04:50.006664 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99038d-f076-4001-9bb9-d42cba2850f7 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:50.017062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99038d-f076-4001-9bb9-d42cba2850f7 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:50.021691 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279591ee-70a0-4787-94fc-ecb65f27a457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:50.026886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279591ee-70a0-4787-94fc-ecb65f27a457 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:50.026886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279591ee-70a0-4787-94fc-ecb65f27a457 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:50.026886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279591ee-70a0-4787-94fc-ecb65f27a457 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:50.026886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279591ee-70a0-4787-94fc-ecb65f27a457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:50.026886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279591ee-70a0-4787-94fc-ecb65f27a457 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:50.040944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1354/2736] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:04:50.041971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf832f0-1b52-4a92-b948-7da968ff4fad admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:50.052420 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf832f0-1b52-4a92-b948-7da968ff4fad admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:50.057047 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b9470-b1bb-4990-92f7-92419bcb0107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:50.057725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b9470-b1bb-4990-92f7-92419bcb0107 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:50.058001 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b9470-b1bb-4990-92f7-92419bcb0107 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:50.058270 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b9470-b1bb-4990-92f7-92419bcb0107 admin admin] PATH_INFO: `/v3/projects/37a5fd8e844c4ea29cbdb7e59281f8ba/users/10d45cbb21264183ad9a835253cd3374/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:50.073429 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b9470-b1bb-4990-92f7-92419bcb0107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a5fd8e844c4ea29cbdb7e59281f8ba, user_id=10d45cbb21264183ad9a835253cd3374, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:50.076190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b9470-b1bb-4990-92f7-92419bcb0107 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:50.148880 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1383/2737] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:04:50 2022] PUT /identity/v3/projects/37a5fd8e844c4ea29cbdb7e59281f8ba/users/10d45cbb21264183ad9a835253cd3374/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:04:50.159265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e929953-d375-4fb1-8bbc-e1700ad88836 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:50.159548 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e929953-d375-4fb1-8bbc-e1700ad88836 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:50.159818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e929953-d375-4fb1-8bbc-e1700ad88836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:50.188220 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2e929953-d375-4fb1-8bbc-e1700ad88836 None None] MFA Rules not processed for user `10d45cbb21264183ad9a835253cd3374`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:50.189443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2e929953-d375-4fb1-8bbc-e1700ad88836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:50.252528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1355/2738] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:50.263053 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b9470-b1bb-4990-92f7-92419bcb0107 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:50.265742 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-398b9470-b1bb-4990-92f7-92419bcb0107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:50.321059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b9470-b1bb-4990-92f7-92419bcb0107 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:50.324708 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7ebaa-923b-4327-ae65-57a1824f84a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:50.325302 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7ebaa-923b-4327-ae65-57a1824f84a4 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:50.325579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7ebaa-923b-4327-ae65-57a1824f84a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:50.325849 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7ebaa-923b-4327-ae65-57a1824f84a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:50.334670 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7ebaa-923b-4327-ae65-57a1824f84a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:50.336114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7ebaa-923b-4327-ae65-57a1824f84a4 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:50.344755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1384/2739] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:51.972800 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb863fd0-03e4-4746-8b92-d1728d465aa1 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:51.973312 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb863fd0-03e4-4746-8b92-d1728d465aa1 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:51.973438 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb863fd0-03e4-4746-8b92-d1728d465aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:51.992441 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-fb863fd0-03e4-4746-8b92-d1728d465aa1 None None] MFA Rules not processed for user `10d45cbb21264183ad9a835253cd3374`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:51.999011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fb863fd0-03e4-4746-8b92-d1728d465aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:52.000854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1356/2740] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:04:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:05:00.340915 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:05:01.986938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:05:13.294355 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7178581-12e8-4bc1-8b70-a07e31ec9f7a None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:13.296674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7178581-12e8-4bc1-8b70-a07e31ec9f7a None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:13.296674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7178581-12e8-4bc1-8b70-a07e31ec9f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:13.323570 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-b7178581-12e8-4bc1-8b70-a07e31ec9f7a None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:05:13.324749 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b7178581-12e8-4bc1-8b70-a07e31ec9f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:13.386000 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1385/2741] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 30 15:05:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:05:13.526575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb863fd0-03e4-4746-8b92-d1728d465aa1 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:13.527986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-fb863fd0-03e4-4746-8b92-d1728d465aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:13.595548 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb863fd0-03e4-4746-8b92-d1728d465aa1 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:13.601328 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92480c2d-bd8c-4099-9322-80fc6c34b8f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:13.601926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92480c2d-bd8c-4099-9322-80fc6c34b8f6 service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:13.602203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92480c2d-bd8c-4099-9322-80fc6c34b8f6 service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:13.602472 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92480c2d-bd8c-4099-9322-80fc6c34b8f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:13.610309 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92480c2d-bd8c-4099-9322-80fc6c34b8f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:13.611923 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92480c2d-bd8c-4099-9322-80fc6c34b8f6 service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:13.619673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1357/2742] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 15:05:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:05:14.414909 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce8619-1b76-4135-a719-ece392d0bf67 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:14.415399 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce8619-1b76-4135-a719-ece392d0bf67 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:14.415494 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce8619-1b76-4135-a719-ece392d0bf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:14.435132 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-8dce8619-1b76-4135-a719-ece392d0bf67 None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:05:14.436326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8dce8619-1b76-4135-a719-ece392d0bf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:14.443495 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1386/2743] 10.222.0.27 () {58 vars in 967 bytes} [Tue Aug 30 15:05:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:05:14.455217 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92480c2d-bd8c-4099-9322-80fc6c34b8f6 service swift] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:14.465522 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92480c2d-bd8c-4099-9322-80fc6c34b8f6 service swift] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:14.470439 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05efb57-677a-4955-9397-41b697d8e982 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:14.471020 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05efb57-677a-4955-9397-41b697d8e982 service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:14.471330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05efb57-677a-4955-9397-41b697d8e982 service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:14.472374 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05efb57-677a-4955-9397-41b697d8e982 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:14.480242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05efb57-677a-4955-9397-41b697d8e982 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:14.481805 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05efb57-677a-4955-9397-41b697d8e982 service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:14.490274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1358/2744] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 15:05:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:05:15.118006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0e5e7-e77d-4a6e-8893-0b818b9f7b11 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:15.118503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0e5e7-e77d-4a6e-8893-0b818b9f7b11 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:15.118938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0e5e7-e77d-4a6e-8893-0b818b9f7b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:15.139667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-7cf0e5e7-e77d-4a6e-8893-0b818b9f7b11 None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:05:15.140900 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7cf0e5e7-e77d-4a6e-8893-0b818b9f7b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:15.148637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1387/2745] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 30 15:05:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:05:15.162250 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05efb57-677a-4955-9397-41b697d8e982 service swift] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:15.174720 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05efb57-677a-4955-9397-41b697d8e982 service swift] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:15.181415 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e96b1d-bf92-453f-87ec-b7dd4b6e0fc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:15.182019 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e96b1d-bf92-453f-87ec-b7dd4b6e0fc4 service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:15.182490 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e96b1d-bf92-453f-87ec-b7dd4b6e0fc4 service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:15.182926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e96b1d-bf92-453f-87ec-b7dd4b6e0fc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:15.192527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e96b1d-bf92-453f-87ec-b7dd4b6e0fc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:15.197089 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e96b1d-bf92-453f-87ec-b7dd4b6e0fc4 service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:15.206088 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1359/2746] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 15:05:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:05:15.848731 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85740c48-1b3e-4448-929f-71f06a53b697 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:15.849192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85740c48-1b3e-4448-929f-71f06a53b697 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:15.849286 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85740c48-1b3e-4448-929f-71f06a53b697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:15.868049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-85740c48-1b3e-4448-929f-71f06a53b697 None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:05:15.869254 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-85740c48-1b3e-4448-929f-71f06a53b697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:15.876482 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1388/2747] 10.222.0.27 () {58 vars in 967 bytes} [Tue Aug 30 15:05:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:05:15.889061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e96b1d-bf92-453f-87ec-b7dd4b6e0fc4 service swift] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:15.900276 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e96b1d-bf92-453f-87ec-b7dd4b6e0fc4 service swift] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:15.905257 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3df3e5-371f-48e6-9082-a3e0ee52c144 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:15.905845 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3df3e5-371f-48e6-9082-a3e0ee52c144 service swift] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:15.906152 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3df3e5-371f-48e6-9082-a3e0ee52c144 service swift] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:15.906424 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3df3e5-371f-48e6-9082-a3e0ee52c144 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:15.914493 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3df3e5-371f-48e6-9082-a3e0ee52c144 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:15.916101 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3df3e5-371f-48e6-9082-a3e0ee52c144 service swift] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:15.924719 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1360/2748] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 15:05:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:05:18.194567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85740c48-1b3e-4448-929f-71f06a53b697 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:18.217971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85740c48-1b3e-4448-929f-71f06a53b697 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:18.222003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869df497-9076-4a14-ab23-72729130c44b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:18.222601 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869df497-9076-4a14-ab23-72729130c44b service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:18.222884 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869df497-9076-4a14-ab23-72729130c44b service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:18.223194 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869df497-9076-4a14-ab23-72729130c44b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:18.224906 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-869df497-9076-4a14-ab23-72729130c44b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:18.287967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869df497-9076-4a14-ab23-72729130c44b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:18.289391 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869df497-9076-4a14-ab23-72729130c44b service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:18.298449 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1389/2749] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:05:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:05:18.637112 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3df3e5-371f-48e6-9082-a3e0ee52c144 service swift] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:18.703594 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3df3e5-371f-48e6-9082-a3e0ee52c144 service swift] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:18.708266 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a870c-eaad-4cfe-a5f7-36e6a05661b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:18.708888 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a870c-eaad-4cfe-a5f7-36e6a05661b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:18.709162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a870c-eaad-4cfe-a5f7-36e6a05661b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:18.709482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a870c-eaad-4cfe-a5f7-36e6a05661b2 admin admin] PATH_INFO: `/v3/users/10d45cbb21264183ad9a835253cd3374` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:18.710651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a870c-eaad-4cfe-a5f7-36e6a05661b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10d45cbb21264183ad9a835253cd3374)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:18.712585 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a870c-eaad-4cfe-a5f7-36e6a05661b2 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:18.879516 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b24a870c-eaad-4cfe-a5f7-36e6a05661b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10d45cbb21264183ad9a835253cd3374'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:18.884449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:05:18.887765 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b24a870c-eaad-4cfe-a5f7-36e6a05661b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10d45cbb21264183ad9a835253cd3374'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:18.888428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b24a870c-eaad-4cfe-a5f7-36e6a05661b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10d45cbb21264183ad9a835253cd3374'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:18.985734 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b24a870c-eaad-4cfe-a5f7-36e6a05661b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10d45cbb21264183ad9a835253cd3374'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:18.999423 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-b24a870c-eaad-4cfe-a5f7-36e6a05661b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10d45cbb21264183ad9a835253cd3374'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:19.005562 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1361/2750] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:05:18 2022] DELETE /identity/v3/users/10d45cbb21264183ad9a835253cd3374 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:05:19.009982 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbad0f9-4715-4aae-8c69-bf3fb0d92059 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:19.010263 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbad0f9-4715-4aae-8c69-bf3fb0d92059 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:19.010532 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbad0f9-4715-4aae-8c69-bf3fb0d92059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:19.029534 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-9fbad0f9-4715-4aae-8c69-bf3fb0d92059 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:05:19.030720 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9fbad0f9-4715-4aae-8c69-bf3fb0d92059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:19.138228 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:05:19.146256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1390/2751] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:05:19.156096 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a870c-eaad-4cfe-a5f7-36e6a05661b2 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:19.158075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-b24a870c-eaad-4cfe-a5f7-36e6a05661b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:19.241957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a870c-eaad-4cfe-a5f7-36e6a05661b2 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:19.245846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349af39-0c8d-4513-b16e-099a04b13524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:19.246442 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349af39-0c8d-4513-b16e-099a04b13524 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:19.246903 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349af39-0c8d-4513-b16e-099a04b13524 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:19.247162 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349af39-0c8d-4513-b16e-099a04b13524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:19.256385 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b349af39-0c8d-4513-b16e-099a04b13524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:19.262304 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b349af39-0c8d-4513-b16e-099a04b13524 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:19.272483 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1362/2752] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:05:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:05:19.479780 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbad0f9-4715-4aae-8c69-bf3fb0d92059 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:19.481347 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-9fbad0f9-4715-4aae-8c69-bf3fb0d92059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:19.493438 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbad0f9-4715-4aae-8c69-bf3fb0d92059 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:19.498247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db2dd8-64ed-4550-873f-6eaf2a9e8c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:19.498881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db2dd8-64ed-4550-873f-6eaf2a9e8c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:19.499190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db2dd8-64ed-4550-873f-6eaf2a9e8c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:19.499485 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db2dd8-64ed-4550-873f-6eaf2a9e8c12 admin admin] PATH_INFO: `/v3/projects/37a5fd8e844c4ea29cbdb7e59281f8ba` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:19.500593 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db2dd8-64ed-4550-873f-6eaf2a9e8c12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37a5fd8e844c4ea29cbdb7e59281f8ba)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:19.502522 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db2dd8-64ed-4550-873f-6eaf2a9e8c12 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:19.626421 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c3db2dd8-64ed-4550-873f-6eaf2a9e8c12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37a5fd8e844c4ea29cbdb7e59281f8ba', 'request_id': 'req-c3db2dd8-64ed-4550-873f-6eaf2a9e8c12'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:19.632670 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c3db2dd8-64ed-4550-873f-6eaf2a9e8c12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37a5fd8e844c4ea29cbdb7e59281f8ba', 'request_id': 'req-c3db2dd8-64ed-4550-873f-6eaf2a9e8c12'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:19.711143 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c3db2dd8-64ed-4550-873f-6eaf2a9e8c12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37a5fd8e844c4ea29cbdb7e59281f8ba', 'request_id': 'req-c3db2dd8-64ed-4550-873f-6eaf2a9e8c12'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:19.728273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c3db2dd8-64ed-4550-873f-6eaf2a9e8c12 admin admin] The token cache is being invalidate because project 37a5fd8e844c4ea29cbdb7e59281f8ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:05:19.728273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c3db2dd8-64ed-4550-873f-6eaf2a9e8c12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:19.728273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c3db2dd8-64ed-4550-873f-6eaf2a9e8c12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:19.728273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1391/2753] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:05:19 2022] DELETE /identity/v3/projects/37a5fd8e844c4ea29cbdb7e59281f8ba => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:05:19.733033 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45bede-c688-4981-b2d4-efc4400fa34d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:19.733488 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45bede-c688-4981-b2d4-efc4400fa34d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:19.733583 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45bede-c688-4981-b2d4-efc4400fa34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:19.752615 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6a45bede-c688-4981-b2d4-efc4400fa34d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:05:19.754332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6a45bede-c688-4981-b2d4-efc4400fa34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:19.811976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1363/2754] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:05:19.821066 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6efb6b-6df8-4ed0-84ab-26ac78032774 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:19.821347 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6efb6b-6df8-4ed0-84ab-26ac78032774 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:19.821643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6efb6b-6df8-4ed0-84ab-26ac78032774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:19.841467 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-6a6efb6b-6df8-4ed0-84ab-26ac78032774 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:05:19.842655 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6a6efb6b-6df8-4ed0-84ab-26ac78032774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:19.852591 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1392/2755] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:05:19.856875 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45bede-c688-4981-b2d4-efc4400fa34d None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:19.866602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45bede-c688-4981-b2d4-efc4400fa34d None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:19.871158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29897cd5-7ad2-4b3d-85eb-2fc6e24ef650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:19.871747 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29897cd5-7ad2-4b3d-85eb-2fc6e24ef650 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:19.872025 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29897cd5-7ad2-4b3d-85eb-2fc6e24ef650 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:19.872310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29897cd5-7ad2-4b3d-85eb-2fc6e24ef650 admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:19.872941 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29897cd5-7ad2-4b3d-85eb-2fc6e24ef650 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:05:19.873244 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29897cd5-7ad2-4b3d-85eb-2fc6e24ef650 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:19.874868 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29897cd5-7ad2-4b3d-85eb-2fc6e24ef650 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:19.886478 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1364/2756] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:05:19.890536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6efb6b-6df8-4ed0-84ab-26ac78032774 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:19.900416 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6efb6b-6df8-4ed0-84ab-26ac78032774 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:19.904966 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68e8b7-37f0-409d-a054-942cfdfac007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:19.905543 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68e8b7-37f0-409d-a054-942cfdfac007 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:19.905841 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68e8b7-37f0-409d-a054-942cfdfac007 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:19.906107 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68e8b7-37f0-409d-a054-942cfdfac007 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:19.906724 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68e8b7-37f0-409d-a054-942cfdfac007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:19.908637 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68e8b7-37f0-409d-a054-942cfdfac007 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:19.998658 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1393/2757] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:05:19 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:05:20.003750 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29897cd5-7ad2-4b3d-85eb-2fc6e24ef650 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:20.013782 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29897cd5-7ad2-4b3d-85eb-2fc6e24ef650 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:20.051202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:05:20.069997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ba23c-2cfb-439b-aa43-96158220e3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:20.070603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ba23c-2cfb-439b-aa43-96158220e3fc admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:20.071235 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ba23c-2cfb-439b-aa43-96158220e3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:20.071694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ba23c-2cfb-439b-aa43-96158220e3fc admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:20.072916 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ba23c-2cfb-439b-aa43-96158220e3fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:20.074909 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ba23c-2cfb-439b-aa43-96158220e3fc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:20.156969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1365/2758] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:05:20 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:05:20.161226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68e8b7-37f0-409d-a054-942cfdfac007 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:20.173054 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68e8b7-37f0-409d-a054-942cfdfac007 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:20.178235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7243d04-a427-413c-ae81-28fca72d2aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:20.178867 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7243d04-a427-413c-ae81-28fca72d2aac admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:20.179171 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7243d04-a427-413c-ae81-28fca72d2aac admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:20.179441 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7243d04-a427-413c-ae81-28fca72d2aac admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:20.180078 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7243d04-a427-413c-ae81-28fca72d2aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:20.181773 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7243d04-a427-413c-ae81-28fca72d2aac admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:20.192314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1394/2759] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:05:20.196462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ba23c-2cfb-439b-aa43-96158220e3fc admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:20.206575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ba23c-2cfb-439b-aa43-96158220e3fc admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:20.211575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72371b0-488a-4dd5-b23e-e64ec62a6a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:20.212203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72371b0-488a-4dd5-b23e-e64ec62a6a77 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:20.212480 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72371b0-488a-4dd5-b23e-e64ec62a6a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:20.212750 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72371b0-488a-4dd5-b23e-e64ec62a6a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:20.213551 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72371b0-488a-4dd5-b23e-e64ec62a6a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:20.215385 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72371b0-488a-4dd5-b23e-e64ec62a6a77 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:20.225742 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1366/2760] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:05:20.229776 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7243d04-a427-413c-ae81-28fca72d2aac admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:20.241572 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7243d04-a427-413c-ae81-28fca72d2aac admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:20.246146 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a42f7d-fbc2-4cb9-a42d-5b2cd4786faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:20.246786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a42f7d-fbc2-4cb9-a42d-5b2cd4786faf admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:20.247058 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a42f7d-fbc2-4cb9-a42d-5b2cd4786faf admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:20.247512 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a42f7d-fbc2-4cb9-a42d-5b2cd4786faf admin admin] PATH_INFO: `/v3/projects/1fac10aecb7b4050ac8c85a40db19d55/users/dbaf48d78bc143d1b3884816a6f3c3f7/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:20.262552 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a42f7d-fbc2-4cb9-a42d-5b2cd4786faf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fac10aecb7b4050ac8c85a40db19d55, user_id=dbaf48d78bc143d1b3884816a6f3c3f7, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:20.265211 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a42f7d-fbc2-4cb9-a42d-5b2cd4786faf admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:20.339921 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1395/2761] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:05:20 2022] PUT /identity/v3/projects/1fac10aecb7b4050ac8c85a40db19d55/users/dbaf48d78bc143d1b3884816a6f3c3f7/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:05:20.344795 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153c765-615f-4835-a9c4-86e11d882d60 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:20.345074 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153c765-615f-4835-a9c4-86e11d882d60 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:20.345342 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153c765-615f-4835-a9c4-86e11d882d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:20.365165 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4153c765-615f-4835-a9c4-86e11d882d60 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:05:20.366345 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4153c765-615f-4835-a9c4-86e11d882d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:20.421658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1367/2762] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:05:20.431289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a42f7d-fbc2-4cb9-a42d-5b2cd4786faf admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:20.432968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d0a42f7d-fbc2-4cb9-a42d-5b2cd4786faf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:20.488491 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a42f7d-fbc2-4cb9-a42d-5b2cd4786faf admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:20.492167 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fd137-8eea-46d3-b44a-a4d9267e999a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:20.492755 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fd137-8eea-46d3-b44a-a4d9267e999a service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:20.493028 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fd137-8eea-46d3-b44a-a4d9267e999a service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:20.493295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fd137-8eea-46d3-b44a-a4d9267e999a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:20.502010 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fd137-8eea-46d3-b44a-a4d9267e999a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:20.503492 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fd137-8eea-46d3-b44a-a4d9267e999a service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:20.512663 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1396/2763] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:05:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:05:26.216828 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b0ca9-af33-41e9-a861-b71676aaecb5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:26.217467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b0ca9-af33-41e9-a861-b71676aaecb5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:26.217467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b0ca9-af33-41e9-a861-b71676aaecb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:26.257183 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4c1b0ca9-af33-41e9-a861-b71676aaecb5 None None] MFA Rules not processed for user `dbaf48d78bc143d1b3884816a6f3c3f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:05:26.258485 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4c1b0ca9-af33-41e9-a861-b71676aaecb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:26.343923 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1368/2764] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:05:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:05:26.349120 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fd137-8eea-46d3-b44a-a4d9267e999a service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:26.358981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fd137-8eea-46d3-b44a-a4d9267e999a service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:26.363671 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54803004-977e-41f8-b290-c5e48196dd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:26.371086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54803004-977e-41f8-b290-c5e48196dd2d admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:26.371086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54803004-977e-41f8-b290-c5e48196dd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:26.371086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54803004-977e-41f8-b290-c5e48196dd2d admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:26.371086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54803004-977e-41f8-b290-c5e48196dd2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:26.371086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54803004-977e-41f8-b290-c5e48196dd2d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:26.501665 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:05:26.515321 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1397/2765] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:05:26 2022] POST /identity/v3/projects => generated 371 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:05:26.515930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b0ca9-af33-41e9-a861-b71676aaecb5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:26.534101 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b0ca9-af33-41e9-a861-b71676aaecb5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:26.587369 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:05:26.599479 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d498f4-f64b-4fa2-a1eb-b23c3c0ead28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:26.600105 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d498f4-f64b-4fa2-a1eb-b23c3c0ead28 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:26.600437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d498f4-f64b-4fa2-a1eb-b23c3c0ead28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:26.600986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d498f4-f64b-4fa2-a1eb-b23c3c0ead28 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:26.601419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d498f4-f64b-4fa2-a1eb-b23c3c0ead28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:26.603408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d498f4-f64b-4fa2-a1eb-b23c3c0ead28 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:26.696309 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1369/2766] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:05:26 2022] POST /identity/v3/users => generated 292 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:05:26.700328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54803004-977e-41f8-b290-c5e48196dd2d admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:26.710941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54803004-977e-41f8-b290-c5e48196dd2d admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:26.715660 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197c95b-1bb4-41de-aa99-3608580fd6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:26.720432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197c95b-1bb4-41de-aa99-3608580fd6de admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:26.720432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197c95b-1bb4-41de-aa99-3608580fd6de admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:26.720432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197c95b-1bb4-41de-aa99-3608580fd6de admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:26.720432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197c95b-1bb4-41de-aa99-3608580fd6de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:26.720432 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197c95b-1bb4-41de-aa99-3608580fd6de admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:26.729848 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1398/2767] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:05:26.733862 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d498f4-f64b-4fa2-a1eb-b23c3c0ead28 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:26.744340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d498f4-f64b-4fa2-a1eb-b23c3c0ead28 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:26.748983 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dfca5-f9c8-48e7-9f00-7a67f3b2d4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:26.753298 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dfca5-f9c8-48e7-9f00-7a67f3b2d4e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:26.753298 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dfca5-f9c8-48e7-9f00-7a67f3b2d4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:26.753298 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dfca5-f9c8-48e7-9f00-7a67f3b2d4e3 admin admin] PATH_INFO: `/v3/projects/1965018347464ce8b4c0f9e8923b81b1/users/aae21d12792e471e8e20adecd8482b14/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:26.767512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16dfca5-f9c8-48e7-9f00-7a67f3b2d4e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1965018347464ce8b4c0f9e8923b81b1, user_id=aae21d12792e471e8e20adecd8482b14, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:26.770263 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16dfca5-f9c8-48e7-9f00-7a67f3b2d4e3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:26.865542 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1370/2768] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:05:26 2022] PUT /identity/v3/projects/1965018347464ce8b4c0f9e8923b81b1/users/aae21d12792e471e8e20adecd8482b14/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:05:26.869480 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197c95b-1bb4-41de-aa99-3608580fd6de admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:26.880193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197c95b-1bb4-41de-aa99-3608580fd6de admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:26.935005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60456c20-fff3-47d2-bb26-a2df6c9db180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:26.935636 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60456c20-fff3-47d2-bb26-a2df6c9db180 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:26.935941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60456c20-fff3-47d2-bb26-a2df6c9db180 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:26.936219 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60456c20-fff3-47d2-bb26-a2df6c9db180 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:26.936897 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60456c20-fff3-47d2-bb26-a2df6c9db180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:26.938533 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60456c20-fff3-47d2-bb26-a2df6c9db180 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:26.948812 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1399/2769] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:05:26.959354 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dfca5-f9c8-48e7-9f00-7a67f3b2d4e3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:26.970619 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dfca5-f9c8-48e7-9f00-7a67f3b2d4e3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:26.976232 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6e9b6-c773-40d9-a46e-0353733f5bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:26.976900 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6e9b6-c773-40d9-a46e-0353733f5bdd admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:26.985612 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6e9b6-c773-40d9-a46e-0353733f5bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:26.985612 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6e9b6-c773-40d9-a46e-0353733f5bdd admin admin] PATH_INFO: `/v3/projects/1965018347464ce8b4c0f9e8923b81b1/users/aae21d12792e471e8e20adecd8482b14/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:26.985612 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b6e9b6-c773-40d9-a46e-0353733f5bdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1965018347464ce8b4c0f9e8923b81b1, user_id=aae21d12792e471e8e20adecd8482b14, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:26.985612 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b6e9b6-c773-40d9-a46e-0353733f5bdd admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:26.993550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1371/2770] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:05:26 2022] PUT /identity/v3/projects/1965018347464ce8b4c0f9e8923b81b1/users/aae21d12792e471e8e20adecd8482b14/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:05:27.140925 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee7f63-1cc9-430f-94dc-c0ee95aa2883 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:27.141870 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee7f63-1cc9-430f-94dc-c0ee95aa2883 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:27.142309 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee7f63-1cc9-430f-94dc-c0ee95aa2883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:27.165824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-a6ee7f63-1cc9-430f-94dc-c0ee95aa2883 None None] MFA Rules not processed for user `c323addb0b2d45ffb8568d6ebfb4a9eb`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:05:27.167069 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a6ee7f63-1cc9-430f-94dc-c0ee95aa2883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:27.185608 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef088bc-ae2e-4237-9fe1-8a34ab6dfbcd None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:27.185854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef088bc-ae2e-4237-9fe1-8a34ab6dfbcd None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:27.186122 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef088bc-ae2e-4237-9fe1-8a34ab6dfbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:27.208422 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-eef088bc-ae2e-4237-9fe1-8a34ab6dfbcd None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:05:27.209724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-eef088bc-ae2e-4237-9fe1-8a34ab6dfbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:27.244721 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1400/2771] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:05:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:05:27.286890 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1372/2772] 10.222.0.27 () {58 vars in 967 bytes} [Tue Aug 30 15:05:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:05:27.299784 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee7f63-1cc9-430f-94dc-c0ee95aa2883 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:27.301817 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a6ee7f63-1cc9-430f-94dc-c0ee95aa2883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:27.412679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee7f63-1cc9-430f-94dc-c0ee95aa2883 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:27.418621 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f392f5-338d-4e61-8a9f-ce932328df94 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:27.419279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f392f5-338d-4e61-8a9f-ce932328df94 service swift] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:27.419557 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f392f5-338d-4e61-8a9f-ce932328df94 service swift] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:27.419826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f392f5-338d-4e61-8a9f-ce932328df94 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:27.438909 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f392f5-338d-4e61-8a9f-ce932328df94 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:27.438909 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f392f5-338d-4e61-8a9f-ce932328df94 service swift] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:27.452920 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1401/2773] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 15:05:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:05:35.130688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f7493-aa18-448b-a199-e7fceb2555c8 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:35.131668 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f7493-aa18-448b-a199-e7fceb2555c8 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:35.132362 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f7493-aa18-448b-a199-e7fceb2555c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:35.169727 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-959f7493-aa18-448b-a199-e7fceb2555c8 None None] MFA Rules not processed for user `aae21d12792e471e8e20adecd8482b14`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:05:35.171538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-959f7493-aa18-448b-a199-e7fceb2555c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:35.269957 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1373/2774] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:05:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:05:35.283301 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f392f5-338d-4e61-8a9f-ce932328df94 service swift] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:35.285048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-85f392f5-338d-4e61-8a9f-ce932328df94 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:35.294853 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f7493-aa18-448b-a199-e7fceb2555c8 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:35.296500 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-959f7493-aa18-448b-a199-e7fceb2555c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:35.318760 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:05:35.334828 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:05:35.387283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f7493-aa18-448b-a199-e7fceb2555c8 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:35.392949 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5e2bb-6e61-404c-ace4-adb20f60b3c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4f11655cd51438e9652869f74ffa975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:35.393633 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5e2bb-6e61-404c-ace4-adb20f60b3c5 service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:35.393919 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5e2bb-6e61-404c-ace4-adb20f60b3c5 service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:35.394271 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5e2bb-6e61-404c-ace4-adb20f60b3c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:35.396398 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-87f5e2bb-6e61-404c-ace4-adb20f60b3c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:35.409038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5e2bb-6e61-404c-ace4-adb20f60b3c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:35.411243 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5e2bb-6e61-404c-ace4-adb20f60b3c5 service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:35.413150 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f392f5-338d-4e61-8a9f-ce932328df94 service swift] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:35.418560 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50fb12-117b-40a9-b5a6-82158819e639 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:35.419361 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50fb12-117b-40a9-b5a6-82158819e639 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:35.419715 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50fb12-117b-40a9-b5a6-82158819e639 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:35.420061 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50fb12-117b-40a9-b5a6-82158819e639 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:35.431704 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1374/2775] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:05:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:05:35.488527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50fb12-117b-40a9-b5a6-82158819e639 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:35.490169 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50fb12-117b-40a9-b5a6-82158819e639 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:35.501301 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1402/2776] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:05:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:05:38.435832 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad64ee2-5f76-4e9d-ad00-8dce9780795b None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:38.436346 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad64ee2-5f76-4e9d-ad00-8dce9780795b None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:38.436449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad64ee2-5f76-4e9d-ad00-8dce9780795b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:38.464745 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-cad64ee2-5f76-4e9d-ad00-8dce9780795b None None] MFA Rules not processed for user `dbaf48d78bc143d1b3884816a6f3c3f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:05:38.474566 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-cad64ee2-5f76-4e9d-ad00-8dce9780795b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:38.484260 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1375/2777] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:05:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:05:40.897628 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50fb12-117b-40a9-b5a6-82158819e639 service nova] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:40.899080 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-6e50fb12-117b-40a9-b5a6-82158819e639 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:41.046073 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50fb12-117b-40a9-b5a6-82158819e639 service nova] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:41.064915 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9527c-e5f3-4e87-88ef-c04cce9334b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:41.066656 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9527c-e5f3-4e87-88ef-c04cce9334b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:41.066953 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9527c-e5f3-4e87-88ef-c04cce9334b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:41.067269 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9527c-e5f3-4e87-88ef-c04cce9334b4 admin admin] PATH_INFO: `/v3/users/2bc1bd791342420c99321b3ebd73bd57` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:41.069415 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9527c-e5f3-4e87-88ef-c04cce9334b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bc1bd791342420c99321b3ebd73bd57)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:41.072266 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9527c-e5f3-4e87-88ef-c04cce9334b4 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:41.322791 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:05:41.339037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c0c9527c-e5f3-4e87-88ef-c04cce9334b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bc1bd791342420c99321b3ebd73bd57'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:41.348405 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c0c9527c-e5f3-4e87-88ef-c04cce9334b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bc1bd791342420c99321b3ebd73bd57'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:41.364235 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c0c9527c-e5f3-4e87-88ef-c04cce9334b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bc1bd791342420c99321b3ebd73bd57'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:41.365274 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c0c9527c-e5f3-4e87-88ef-c04cce9334b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bc1bd791342420c99321b3ebd73bd57'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:41.366138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c0c9527c-e5f3-4e87-88ef-c04cce9334b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bc1bd791342420c99321b3ebd73bd57'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:41.488676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1403/2778] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:05:40 2022] DELETE /identity/v3/users/2bc1bd791342420c99321b3ebd73bd57 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:05:41.758477 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad64ee2-5f76-4e9d-ad00-8dce9780795b None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:41.770776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad64ee2-5f76-4e9d-ad00-8dce9780795b None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:41.838236 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:41.838236 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:41.838236 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:41.838236 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4 admin admin] PATH_INFO: `/v3/projects/aa0df72533d34237a9b1868391740224` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:41.838713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa0df72533d34237a9b1868391740224)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:41.839651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:42.004204 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa0df72533d34237a9b1868391740224', 'request_id': 'req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:42.010526 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa0df72533d34237a9b1868391740224', 'request_id': 'req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:42.017234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa0df72533d34237a9b1868391740224', 'request_id': 'req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:42.113463 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4 admin admin] The token cache is being invalidate because project aa0df72533d34237a9b1868391740224 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:05:42.113818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:42.114834 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:42.116702 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1376/2779] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:05:41 2022] DELETE /identity/v3/projects/aa0df72533d34237a9b1868391740224 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:05:42.120720 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9527c-e5f3-4e87-88ef-c04cce9334b4 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:42.123577 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c0c9527c-e5f3-4e87-88ef-c04cce9334b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:42.139313 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9527c-e5f3-4e87-88ef-c04cce9334b4 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:42.198664 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c51f1-6a73-4012-949e-7a07684d6f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:42.199394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c51f1-6a73-4012-949e-7a07684d6f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:42.199672 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c51f1-6a73-4012-949e-7a07684d6f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:42.199941 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c51f1-6a73-4012-949e-7a07684d6f09 admin admin] PATH_INFO: `/v3/users/643bb62a05e64c6c844fda8ec2ee7e41` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:42.201350 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c51f1-6a73-4012-949e-7a07684d6f09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=643bb62a05e64c6c844fda8ec2ee7e41)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:42.203359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c51f1-6a73-4012-949e-7a07684d6f09 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:42.387569 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-eb6c51f1-6a73-4012-949e-7a07684d6f09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '643bb62a05e64c6c844fda8ec2ee7e41'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:42.394403 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-eb6c51f1-6a73-4012-949e-7a07684d6f09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '643bb62a05e64c6c844fda8ec2ee7e41'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:42.406614 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-eb6c51f1-6a73-4012-949e-7a07684d6f09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '643bb62a05e64c6c844fda8ec2ee7e41'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:42.407399 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-eb6c51f1-6a73-4012-949e-7a07684d6f09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '643bb62a05e64c6c844fda8ec2ee7e41'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:42.408014 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-eb6c51f1-6a73-4012-949e-7a07684d6f09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '643bb62a05e64c6c844fda8ec2ee7e41'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:42.478181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1404/2780] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:05:42 2022] DELETE /identity/v3/users/643bb62a05e64c6c844fda8ec2ee7e41 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:05:42.787586 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:42.789660 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:42.803630 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9ac9c-a288-4f0a-8c53-5979aeca7cd4 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:42.848920 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:05:42.864194 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3ced6-cbcb-4d8c-9197-b58c63ea03df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:42.864818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3ced6-cbcb-4d8c-9197-b58c63ea03df admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:42.865094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3ced6-cbcb-4d8c-9197-b58c63ea03df admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:42.865363 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3ced6-cbcb-4d8c-9197-b58c63ea03df admin admin] PATH_INFO: `/v3/projects/dcca2432532f4d08b391afbb7f15cf6a` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:42.866478 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3ced6-cbcb-4d8c-9197-b58c63ea03df admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcca2432532f4d08b391afbb7f15cf6a)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:42.868409 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3ced6-cbcb-4d8c-9197-b58c63ea03df admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:42.990755 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-76c3ced6-cbcb-4d8c-9197-b58c63ea03df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcca2432532f4d08b391afbb7f15cf6a', 'request_id': 'req-76c3ced6-cbcb-4d8c-9197-b58c63ea03df'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:42.997373 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-76c3ced6-cbcb-4d8c-9197-b58c63ea03df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcca2432532f4d08b391afbb7f15cf6a', 'request_id': 'req-76c3ced6-cbcb-4d8c-9197-b58c63ea03df'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:43.003457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-76c3ced6-cbcb-4d8c-9197-b58c63ea03df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcca2432532f4d08b391afbb7f15cf6a', 'request_id': 'req-76c3ced6-cbcb-4d8c-9197-b58c63ea03df'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:43.087982 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-76c3ced6-cbcb-4d8c-9197-b58c63ea03df admin admin] The token cache is being invalidate because project dcca2432532f4d08b391afbb7f15cf6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:05:43.088336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-76c3ced6-cbcb-4d8c-9197-b58c63ea03df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:43.089203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-76c3ced6-cbcb-4d8c-9197-b58c63ea03df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:05:43.090723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1377/2781] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:05:42 2022] DELETE /identity/v3/projects/dcca2432532f4d08b391afbb7f15cf6a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:05:50.131945 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c51f1-6a73-4012-949e-7a07684d6f09 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:50.136661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-eb6c51f1-6a73-4012-949e-7a07684d6f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:50.175649 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:05:50.193019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c51f1-6a73-4012-949e-7a07684d6f09 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:50.241180 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02698315-7c21-47bd-853b-dd19131b85aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:50.241763 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02698315-7c21-47bd-853b-dd19131b85aa service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:50.242038 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02698315-7c21-47bd-853b-dd19131b85aa service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:50.242308 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02698315-7c21-47bd-853b-dd19131b85aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:50.244054 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-02698315-7c21-47bd-853b-dd19131b85aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:50.307057 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02698315-7c21-47bd-853b-dd19131b85aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:50.308964 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02698315-7c21-47bd-853b-dd19131b85aa service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:50.318788 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1405/2782] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:05:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:05:53.084702 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:05:54.015587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849390a-b118-4caa-9e83-ee2ebeaeb347 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:54.015876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849390a-b118-4caa-9e83-ee2ebeaeb347 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:54.016194 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849390a-b118-4caa-9e83-ee2ebeaeb347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:54.035829 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a849390a-b118-4caa-9e83-ee2ebeaeb347 None None] MFA Rules not processed for user `6a9784d49f8d45e38f096de3b2401d36`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:05:54.037570 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a849390a-b118-4caa-9e83-ee2ebeaeb347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:54.093553 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1378/2783] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:05:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:05:54.407944 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02698315-7c21-47bd-853b-dd19131b85aa service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:54.409648 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-02698315-7c21-47bd-853b-dd19131b85aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:54.421696 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02698315-7c21-47bd-853b-dd19131b85aa service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:54.426835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f7bbf-d5da-4fe2-a78f-0d19af7888b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:54.427579 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f7bbf-d5da-4fe2-a78f-0d19af7888b8 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:54.427970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f7bbf-d5da-4fe2-a78f-0d19af7888b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:54.428280 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f7bbf-d5da-4fe2-a78f-0d19af7888b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:54.430216 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ef4f7bbf-d5da-4fe2-a78f-0d19af7888b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:54.441602 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f7bbf-d5da-4fe2-a78f-0d19af7888b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:54.443214 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f7bbf-d5da-4fe2-a78f-0d19af7888b8 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:54.452648 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1406/2784] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:05:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:06:04.086818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=25 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:06:04.447692 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=60 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:06:15.645943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849390a-b118-4caa-9e83-ee2ebeaeb347 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:06:15.655969 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849390a-b118-4caa-9e83-ee2ebeaeb347 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:06:15.659480 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdf827-3ec9-4aff-ba9f-5eb97be810be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:06:15.660126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdf827-3ec9-4aff-ba9f-5eb97be810be service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:06:15.660404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdf827-3ec9-4aff-ba9f-5eb97be810be service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:06:15.660671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdf827-3ec9-4aff-ba9f-5eb97be810be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:06:15.662392 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6bfdf827-3ec9-4aff-ba9f-5eb97be810be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:06:15.725349 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdf827-3ec9-4aff-ba9f-5eb97be810be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:06:15.726778 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdf827-3ec9-4aff-ba9f-5eb97be810be service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:06:15.736725 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1379/2785] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:06:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:06:25.732470 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=30 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:06:49.241224 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f7bbf-d5da-4fe2-a78f-0d19af7888b8 service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:06:49.244969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ef4f7bbf-d5da-4fe2-a78f-0d19af7888b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:06:49.313701 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f7bbf-d5da-4fe2-a78f-0d19af7888b8 service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:06:49.317328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc13b27-a06e-459d-97c4-041f6079638d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:06:49.317983 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc13b27-a06e-459d-97c4-041f6079638d service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:06:49.318267 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc13b27-a06e-459d-97c4-041f6079638d service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:06:49.318539 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc13b27-a06e-459d-97c4-041f6079638d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:06:49.320275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-bcc13b27-a06e-459d-97c4-041f6079638d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:06:49.389431 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc13b27-a06e-459d-97c4-041f6079638d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:06:49.390769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc13b27-a06e-459d-97c4-041f6079638d service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:06:49.402041 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1407/2786] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:06:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:06:57.725757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdf827-3ec9-4aff-ba9f-5eb97be810be service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:06:57.735194 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdf827-3ec9-4aff-ba9f-5eb97be810be service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:06:57.738776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761714b-786f-4e45-a3a4-cb658460aa97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:06:57.739397 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761714b-786f-4e45-a3a4-cb658460aa97 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:06:57.739675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761714b-786f-4e45-a3a4-cb658460aa97 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:06:57.739941 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761714b-786f-4e45-a3a4-cb658460aa97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:06:57.741631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9761714b-786f-4e45-a3a4-cb658460aa97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:06:57.754607 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9761714b-786f-4e45-a3a4-cb658460aa97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:06:57.756062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9761714b-786f-4e45-a3a4-cb658460aa97 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:06:57.767336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1380/2787] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:06:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:06:59.396626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=59 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:07:07.271370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26543599-3ed1-404e-aae9-5c3b95cab45b None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:07.275454 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26543599-3ed1-404e-aae9-5c3b95cab45b None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:07.275454 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26543599-3ed1-404e-aae9-5c3b95cab45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:07.291752 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-26543599-3ed1-404e-aae9-5c3b95cab45b None None] MFA Rules not processed for user `6a9784d49f8d45e38f096de3b2401d36`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:07:07.299329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-26543599-3ed1-404e-aae9-5c3b95cab45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:07.300385 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1408/2788] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:07:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:07:07.762533 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:07:17.286406 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:07:39.669862 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761714b-786f-4e45-a3a4-cb658460aa97 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:39.671839 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9761714b-786f-4e45-a3a4-cb658460aa97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:39.684543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9761714b-786f-4e45-a3a4-cb658460aa97 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:39.688159 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b5bd9-9523-4833-b8bf-396bb8951ab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:39.688864 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b5bd9-9523-4833-b8bf-396bb8951ab1 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:39.689139 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b5bd9-9523-4833-b8bf-396bb8951ab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:39.689405 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b5bd9-9523-4833-b8bf-396bb8951ab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:39.691171 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-860b5bd9-9523-4833-b8bf-396bb8951ab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:39.701774 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b5bd9-9523-4833-b8bf-396bb8951ab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:39.703290 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b5bd9-9523-4833-b8bf-396bb8951ab1 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:39.712111 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1381/2789] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:07:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:07:42.921343 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26543599-3ed1-404e-aae9-5c3b95cab45b None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:42.931309 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26543599-3ed1-404e-aae9-5c3b95cab45b None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:42.934938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d58478-a85e-4dc9-8c6c-4be14cb521dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:42.935563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d58478-a85e-4dc9-8c6c-4be14cb521dd service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:42.935842 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d58478-a85e-4dc9-8c6c-4be14cb521dd service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:42.936115 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d58478-a85e-4dc9-8c6c-4be14cb521dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:42.937849 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-31d58478-a85e-4dc9-8c6c-4be14cb521dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:43.000666 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d58478-a85e-4dc9-8c6c-4be14cb521dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:43.002089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d58478-a85e-4dc9-8c6c-4be14cb521dd service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:43.011716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1409/2790] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:07:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:07:49.708165 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=6 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:07:49.907837 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b5bd9-9523-4833-b8bf-396bb8951ab1 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:49.909275 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-860b5bd9-9523-4833-b8bf-396bb8951ab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:49.924698 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b5bd9-9523-4833-b8bf-396bb8951ab1 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:49.940294 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04916305-a2d5-4000-ace1-dba536d59bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:49.940294 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04916305-a2d5-4000-ace1-dba536d59bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:49.940294 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04916305-a2d5-4000-ace1-dba536d59bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:49.940294 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04916305-a2d5-4000-ace1-dba536d59bc6 admin admin] PATH_INFO: `/v3/users/c323addb0b2d45ffb8568d6ebfb4a9eb` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:49.940774 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04916305-a2d5-4000-ace1-dba536d59bc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c323addb0b2d45ffb8568d6ebfb4a9eb)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:49.940774 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04916305-a2d5-4000-ace1-dba536d59bc6 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:50.174854 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-04916305-a2d5-4000-ace1-dba536d59bc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c323addb0b2d45ffb8568d6ebfb4a9eb'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:50.183205 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-04916305-a2d5-4000-ace1-dba536d59bc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c323addb0b2d45ffb8568d6ebfb4a9eb'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:50.184021 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-04916305-a2d5-4000-ace1-dba536d59bc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c323addb0b2d45ffb8568d6ebfb4a9eb'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:50.275511 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-04916305-a2d5-4000-ace1-dba536d59bc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c323addb0b2d45ffb8568d6ebfb4a9eb'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:50.288654 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-04916305-a2d5-4000-ace1-dba536d59bc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c323addb0b2d45ffb8568d6ebfb4a9eb'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:50.294404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1382/2791] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:07:49 2022] DELETE /identity/v3/users/c323addb0b2d45ffb8568d6ebfb4a9eb => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:07:50.536497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d58478-a85e-4dc9-8c6c-4be14cb521dd service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:50.546384 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d58478-a85e-4dc9-8c6c-4be14cb521dd service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:50.605513 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d1692-7885-4f7d-8e98-0e28092e20d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:50.606158 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d1692-7885-4f7d-8e98-0e28092e20d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:50.606413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d1692-7885-4f7d-8e98-0e28092e20d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:50.606684 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d1692-7885-4f7d-8e98-0e28092e20d9 admin admin] PATH_INFO: `/v3/projects/f4a168c27895481dbc738788feefa2a8` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:50.607827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d1692-7885-4f7d-8e98-0e28092e20d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4a168c27895481dbc738788feefa2a8)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:50.609723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d1692-7885-4f7d-8e98-0e28092e20d9 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:50.726245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a22d1692-7885-4f7d-8e98-0e28092e20d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4a168c27895481dbc738788feefa2a8', 'request_id': 'req-a22d1692-7885-4f7d-8e98-0e28092e20d9'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:50.731687 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a22d1692-7885-4f7d-8e98-0e28092e20d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4a168c27895481dbc738788feefa2a8', 'request_id': 'req-a22d1692-7885-4f7d-8e98-0e28092e20d9'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:50.803443 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a22d1692-7885-4f7d-8e98-0e28092e20d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4a168c27895481dbc738788feefa2a8', 'request_id': 'req-a22d1692-7885-4f7d-8e98-0e28092e20d9'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:50.815084 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a22d1692-7885-4f7d-8e98-0e28092e20d9 admin admin] The token cache is being invalidate because project f4a168c27895481dbc738788feefa2a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:07:50.815392 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a22d1692-7885-4f7d-8e98-0e28092e20d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:50.818380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a22d1692-7885-4f7d-8e98-0e28092e20d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:50.818380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1410/2792] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:07:50 2022] DELETE /identity/v3/projects/f4a168c27895481dbc738788feefa2a8 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:07:50.821576 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04916305-a2d5-4000-ace1-dba536d59bc6 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:50.822954 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-04916305-a2d5-4000-ace1-dba536d59bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:50.835966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04916305-a2d5-4000-ace1-dba536d59bc6 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:50.891194 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69ca47-1367-4b06-ad4e-bf71c1622a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:50.891812 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69ca47-1367-4b06-ad4e-bf71c1622a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:50.892087 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69ca47-1367-4b06-ad4e-bf71c1622a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:50.892356 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69ca47-1367-4b06-ad4e-bf71c1622a42 admin admin] PATH_INFO: `/v3/users/e11a8cab2d4e4db99f0e29ae8831b87b` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:50.893470 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69ca47-1367-4b06-ad4e-bf71c1622a42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e11a8cab2d4e4db99f0e29ae8831b87b)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:50.895410 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69ca47-1367-4b06-ad4e-bf71c1622a42 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:51.088560 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8f69ca47-1367-4b06-ad4e-bf71c1622a42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e11a8cab2d4e4db99f0e29ae8831b87b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:51.095522 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8f69ca47-1367-4b06-ad4e-bf71c1622a42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e11a8cab2d4e4db99f0e29ae8831b87b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:51.096173 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8f69ca47-1367-4b06-ad4e-bf71c1622a42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e11a8cab2d4e4db99f0e29ae8831b87b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:51.194757 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8f69ca47-1367-4b06-ad4e-bf71c1622a42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e11a8cab2d4e4db99f0e29ae8831b87b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:51.203132 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:07:51.207892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8f69ca47-1367-4b06-ad4e-bf71c1622a42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e11a8cab2d4e4db99f0e29ae8831b87b'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:51.214408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1383/2793] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:07:50 2022] DELETE /identity/v3/users/e11a8cab2d4e4db99f0e29ae8831b87b => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:07:51.447558 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d1692-7885-4f7d-8e98-0e28092e20d9 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:51.450520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a22d1692-7885-4f7d-8e98-0e28092e20d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:51.462989 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d1692-7885-4f7d-8e98-0e28092e20d9 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:51.485393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:07:51.520821 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:51.521436 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:51.521709 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:51.522020 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18 admin admin] PATH_INFO: `/v3/projects/21821c00ee6c4b64ac3fa0cf81ef15d6` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:51.523196 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21821c00ee6c4b64ac3fa0cf81ef15d6)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:51.525185 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:51.638725 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21821c00ee6c4b64ac3fa0cf81ef15d6', 'request_id': 'req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:51.644205 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21821c00ee6c4b64ac3fa0cf81ef15d6', 'request_id': 'req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:51.772713 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21821c00ee6c4b64ac3fa0cf81ef15d6', 'request_id': 'req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:51.785147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18 admin admin] The token cache is being invalidate because project 21821c00ee6c4b64ac3fa0cf81ef15d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:07:51.785634 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:51.786262 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:51.787767 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1411/2794] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:07:51 2022] DELETE /identity/v3/projects/21821c00ee6c4b64ac3fa0cf81ef15d6 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:08:01.309199 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:08:01.782161 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:08:11.146600 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69ca47-1367-4b06-ad4e-bf71c1622a42 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:11.148083 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-8f69ca47-1367-4b06-ad4e-bf71c1622a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:11.234547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69ca47-1367-4b06-ad4e-bf71c1622a42 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:11.333914 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9bda7-878d-423d-82f8-79d30e864e99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:11.335228 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9bda7-878d-423d-82f8-79d30e864e99 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:11.335756 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9bda7-878d-423d-82f8-79d30e864e99 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:11.336694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9bda7-878d-423d-82f8-79d30e864e99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:11.339072 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-60b9bda7-878d-423d-82f8-79d30e864e99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:11.360164 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9bda7-878d-423d-82f8-79d30e864e99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:11.365443 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9bda7-878d-423d-82f8-79d30e864e99 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:11.376709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1384/2795] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:08:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:08:17.998858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:18.000349 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:18.058095 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7c5d0-b77d-44cf-aedf-b6e5fbf52c18 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:18.061997 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e09b1-6722-4f65-8125-071a7c0253b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:18.062602 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e09b1-6722-4f65-8125-071a7c0253b0 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:18.062881 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e09b1-6722-4f65-8125-071a7c0253b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:18.063181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e09b1-6722-4f65-8125-071a7c0253b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:18.064902 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ff2e09b1-6722-4f65-8125-071a7c0253b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:18.128951 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e09b1-6722-4f65-8125-071a7c0253b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:18.130390 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e09b1-6722-4f65-8125-071a7c0253b0 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:18.140901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1412/2796] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:08:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:08:21.372221 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=62 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:08:21.633382 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9bda7-878d-423d-82f8-79d30e864e99 service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:21.635500 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-60b9bda7-878d-423d-82f8-79d30e864e99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:21.652961 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9bda7-878d-423d-82f8-79d30e864e99 service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:21.659473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec5108-66e6-4242-b584-b74685c4bfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:21.659473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec5108-66e6-4242-b584-b74685c4bfcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:21.659473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec5108-66e6-4242-b584-b74685c4bfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:21.659473 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec5108-66e6-4242-b584-b74685c4bfcc admin admin] PATH_INFO: `/v3/users/6a9784d49f8d45e38f096de3b2401d36` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:21.660538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec5108-66e6-4242-b584-b74685c4bfcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a9784d49f8d45e38f096de3b2401d36)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:21.662469 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec5108-66e6-4242-b584-b74685c4bfcc admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:21.884141 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-74ec5108-66e6-4242-b584-b74685c4bfcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a9784d49f8d45e38f096de3b2401d36'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:21.891062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-74ec5108-66e6-4242-b584-b74685c4bfcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a9784d49f8d45e38f096de3b2401d36'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:21.891770 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-74ec5108-66e6-4242-b584-b74685c4bfcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a9784d49f8d45e38f096de3b2401d36'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:21.965632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-74ec5108-66e6-4242-b584-b74685c4bfcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a9784d49f8d45e38f096de3b2401d36'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:21.977931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-74ec5108-66e6-4242-b584-b74685c4bfcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a9784d49f8d45e38f096de3b2401d36'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:21.983786 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1385/2797] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:08:21 2022] DELETE /identity/v3/users/6a9784d49f8d45e38f096de3b2401d36 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:08:22.227065 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e09b1-6722-4f65-8125-071a7c0253b0 service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:22.239370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e09b1-6722-4f65-8125-071a7c0253b0 service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:22.302590 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfaf34d-1342-4a11-96f9-677077873322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:22.303249 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaf34d-1342-4a11-96f9-677077873322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:22.303531 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaf34d-1342-4a11-96f9-677077873322 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:22.303831 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaf34d-1342-4a11-96f9-677077873322 admin admin] PATH_INFO: `/v3/projects/74d147248b7c46bb821f0ca4cd7efd0c` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:22.304915 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfaf34d-1342-4a11-96f9-677077873322 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74d147248b7c46bb821f0ca4cd7efd0c)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:22.306845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfaf34d-1342-4a11-96f9-677077873322 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:22.413804 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7dfaf34d-1342-4a11-96f9-677077873322 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74d147248b7c46bb821f0ca4cd7efd0c', 'request_id': 'req-7dfaf34d-1342-4a11-96f9-677077873322'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:22.415676 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:08:22.420094 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7dfaf34d-1342-4a11-96f9-677077873322 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74d147248b7c46bb821f0ca4cd7efd0c', 'request_id': 'req-7dfaf34d-1342-4a11-96f9-677077873322'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:22.510599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7dfaf34d-1342-4a11-96f9-677077873322 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74d147248b7c46bb821f0ca4cd7efd0c', 'request_id': 'req-7dfaf34d-1342-4a11-96f9-677077873322'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:22.521726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7dfaf34d-1342-4a11-96f9-677077873322 admin admin] The token cache is being invalidate because project 74d147248b7c46bb821f0ca4cd7efd0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:08:22.522016 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7dfaf34d-1342-4a11-96f9-677077873322 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:22.522625 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-7dfaf34d-1342-4a11-96f9-677077873322 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:22.524238 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1413/2798] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:08:22 2022] DELETE /identity/v3/projects/74d147248b7c46bb821f0ca4cd7efd0c => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:08:22.529210 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfd3d1-b888-4ccd-8f45-5eaa26b57bcd None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:22.529681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfd3d1-b888-4ccd-8f45-5eaa26b57bcd None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:22.529791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfd3d1-b888-4ccd-8f45-5eaa26b57bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:22.549818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-3cbfd3d1-b888-4ccd-8f45-5eaa26b57bcd None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:08:22.551002 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-3cbfd3d1-b888-4ccd-8f45-5eaa26b57bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:22.609358 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1386/2799] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:08:22.617872 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f0cb2-a926-4b54-9cee-d6449d094577 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:22.618155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f0cb2-a926-4b54-9cee-d6449d094577 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:22.618427 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f0cb2-a926-4b54-9cee-d6449d094577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:22.637638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-b52f0cb2-a926-4b54-9cee-d6449d094577 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:08:22.638835 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b52f0cb2-a926-4b54-9cee-d6449d094577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:22.646655 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1414/2800] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:08:22.650681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfd3d1-b888-4ccd-8f45-5eaa26b57bcd None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:22.660646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfd3d1-b888-4ccd-8f45-5eaa26b57bcd None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:22.665245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da021bd5-493a-4543-bac9-e47d8509a97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:22.665781 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da021bd5-493a-4543-bac9-e47d8509a97b admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:22.666063 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da021bd5-493a-4543-bac9-e47d8509a97b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:22.666333 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da021bd5-493a-4543-bac9-e47d8509a97b admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:22.666978 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da021bd5-493a-4543-bac9-e47d8509a97b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:08:22.667314 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da021bd5-493a-4543-bac9-e47d8509a97b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:22.669554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da021bd5-493a-4543-bac9-e47d8509a97b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:22.681588 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1387/2801] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:08:22.685852 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f0cb2-a926-4b54-9cee-d6449d094577 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:22.695017 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f0cb2-a926-4b54-9cee-d6449d094577 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:22.699710 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2535e7-b5c5-41f7-a8aa-281eff3a0644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:22.700296 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2535e7-b5c5-41f7-a8aa-281eff3a0644 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:22.700571 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2535e7-b5c5-41f7-a8aa-281eff3a0644 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:22.700840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2535e7-b5c5-41f7-a8aa-281eff3a0644 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:22.701494 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2535e7-b5c5-41f7-a8aa-281eff3a0644 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:22.703414 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2535e7-b5c5-41f7-a8aa-281eff3a0644 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:22.804071 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1415/2802] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:08:22 2022] POST /identity/v3/projects => generated 367 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:08:22.805589 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da021bd5-493a-4543-bac9-e47d8509a97b admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:22.816666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da021bd5-493a-4543-bac9-e47d8509a97b admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:22.873212 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03856e68-7935-4ae8-9e96-bc057e4a12b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:22.873817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03856e68-7935-4ae8-9e96-bc057e4a12b9 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:22.874091 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03856e68-7935-4ae8-9e96-bc057e4a12b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:22.874357 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03856e68-7935-4ae8-9e96-bc057e4a12b9 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:22.874983 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03856e68-7935-4ae8-9e96-bc057e4a12b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:22.876921 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03856e68-7935-4ae8-9e96-bc057e4a12b9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:22.884914 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:08:22.942802 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1388/2803] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:08:22 2022] POST /identity/v3/users => generated 284 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:08:22.946927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2535e7-b5c5-41f7-a8aa-281eff3a0644 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:22.957405 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2535e7-b5c5-41f7-a8aa-281eff3a0644 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:22.961999 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a898b6-c6ba-4bdc-9eab-0b08170b0a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:22.962580 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a898b6-c6ba-4bdc-9eab-0b08170b0a07 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:22.962857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a898b6-c6ba-4bdc-9eab-0b08170b0a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:22.963151 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a898b6-c6ba-4bdc-9eab-0b08170b0a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:22.963786 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a898b6-c6ba-4bdc-9eab-0b08170b0a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:22.965928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a898b6-c6ba-4bdc-9eab-0b08170b0a07 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:22.975965 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1416/2804] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:08:22.979799 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03856e68-7935-4ae8-9e96-bc057e4a12b9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:22.990228 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03856e68-7935-4ae8-9e96-bc057e4a12b9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:22.995663 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b0c91-a219-42bd-82c1-fb0e01efdc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:22.996280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b0c91-a219-42bd-82c1-fb0e01efdc45 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:23.005527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b0c91-a219-42bd-82c1-fb0e01efdc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:23.005527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b0c91-a219-42bd-82c1-fb0e01efdc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:23.005527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b0c91-a219-42bd-82c1-fb0e01efdc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:23.005527 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b0c91-a219-42bd-82c1-fb0e01efdc45 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:23.009428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1389/2805] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:08:23.013885 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a898b6-c6ba-4bdc-9eab-0b08170b0a07 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:23.023760 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a898b6-c6ba-4bdc-9eab-0b08170b0a07 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:23.028967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c69f10-7287-4f0e-8172-8b27df838465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:23.030865 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c69f10-7287-4f0e-8172-8b27df838465 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:23.030865 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c69f10-7287-4f0e-8172-8b27df838465 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:23.030865 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c69f10-7287-4f0e-8172-8b27df838465 admin admin] PATH_INFO: `/v3/projects/fad9a3739a7f4b74bdf62645ff77bae2/users/e62c5e6ae98149cf83a6d0d32119e640/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:23.044711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c69f10-7287-4f0e-8172-8b27df838465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fad9a3739a7f4b74bdf62645ff77bae2, user_id=e62c5e6ae98149cf83a6d0d32119e640, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:23.047382 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c69f10-7287-4f0e-8172-8b27df838465 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:23.122250 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1417/2806] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:08:23 2022] PUT /identity/v3/projects/fad9a3739a7f4b74bdf62645ff77bae2/users/e62c5e6ae98149cf83a6d0d32119e640/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:08:23.132617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f80160-27fb-4f0e-b94a-6cf90ec89b2f None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:23.132904 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f80160-27fb-4f0e-b94a-6cf90ec89b2f None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:23.133177 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f80160-27fb-4f0e-b94a-6cf90ec89b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:23.162834 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-e1f80160-27fb-4f0e-b94a-6cf90ec89b2f None None] MFA Rules not processed for user `e62c5e6ae98149cf83a6d0d32119e640`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:08:23.164068 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e1f80160-27fb-4f0e-b94a-6cf90ec89b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:23.226358 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1390/2807] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:08:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:08:23.237245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c69f10-7287-4f0e-8172-8b27df838465 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:23.238930 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-e6c69f10-7287-4f0e-8172-8b27df838465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:23.296363 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c69f10-7287-4f0e-8172-8b27df838465 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:23.299992 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec6f1c-5d72-4a60-a8ed-d2b84ca89302 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:23.300594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec6f1c-5d72-4a60-a8ed-d2b84ca89302 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:23.300868 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec6f1c-5d72-4a60-a8ed-d2b84ca89302 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:23.301137 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec6f1c-5d72-4a60-a8ed-d2b84ca89302 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:23.309267 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec6f1c-5d72-4a60-a8ed-d2b84ca89302 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:23.310685 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec6f1c-5d72-4a60-a8ed-d2b84ca89302 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:23.319295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1418/2808] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:08:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:08:30.418449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f80160-27fb-4f0e-b94a-6cf90ec89b2f None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:30.420277 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-e1f80160-27fb-4f0e-b94a-6cf90ec89b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:30.478791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f80160-27fb-4f0e-b94a-6cf90ec89b2f None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:30.483077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a63c01-a0c6-479a-ae12-c754c0877cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:30.483713 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a63c01-a0c6-479a-ae12-c754c0877cef service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:30.483991 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a63c01-a0c6-479a-ae12-c754c0877cef service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:30.484285 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a63c01-a0c6-479a-ae12-c754c0877cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:30.485987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-96a63c01-a0c6-479a-ae12-c754c0877cef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:30.495721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a63c01-a0c6-479a-ae12-c754c0877cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:30.497802 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a63c01-a0c6-479a-ae12-c754c0877cef service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:30.505550 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1391/2809] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:08:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:08:30.576629 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec6f1c-5d72-4a60-a8ed-d2b84ca89302 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:30.578036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ceec6f1c-5d72-4a60-a8ed-d2b84ca89302 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:30.589830 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec6f1c-5d72-4a60-a8ed-d2b84ca89302 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:30.593607 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656184c-9db2-43c1-9ad0-86f9b7a7be31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:30.594186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656184c-9db2-43c1-9ad0-86f9b7a7be31 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:30.594462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656184c-9db2-43c1-9ad0-86f9b7a7be31 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:30.594734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656184c-9db2-43c1-9ad0-86f9b7a7be31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:30.596421 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d656184c-9db2-43c1-9ad0-86f9b7a7be31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:30.665493 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:08:30.667646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656184c-9db2-43c1-9ad0-86f9b7a7be31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:30.669879 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656184c-9db2-43c1-9ad0-86f9b7a7be31 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:30.681387 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1419/2810] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:08:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:08:30.745419 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a63c01-a0c6-479a-ae12-c754c0877cef service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:30.747487 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-96a63c01-a0c6-479a-ae12-c754c0877cef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:30.816833 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a63c01-a0c6-479a-ae12-c754c0877cef service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:30.821916 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613978ed-74a8-4edd-b96a-17486bb0b44a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4f11655cd51438e9652869f74ffa975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:30.822513 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613978ed-74a8-4edd-b96a-17486bb0b44a service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:30.822786 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613978ed-74a8-4edd-b96a-17486bb0b44a service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:30.823052 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613978ed-74a8-4edd-b96a-17486bb0b44a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:30.824911 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-613978ed-74a8-4edd-b96a-17486bb0b44a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:30.836252 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613978ed-74a8-4edd-b96a-17486bb0b44a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:30.837847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613978ed-74a8-4edd-b96a-17486bb0b44a service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:30.843808 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:08:30.848413 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1392/2811] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:08:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:08:31.509318 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656184c-9db2-43c1-9ad0-86f9b7a7be31 service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:31.511085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-d656184c-9db2-43c1-9ad0-86f9b7a7be31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:31.532972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656184c-9db2-43c1-9ad0-86f9b7a7be31 service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:31.540487 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a13a3a-2bfc-4c01-a231-778e8696b553 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:31.541295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a13a3a-2bfc-4c01-a231-778e8696b553 service nova] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:31.541800 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a13a3a-2bfc-4c01-a231-778e8696b553 service nova] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:31.542203 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a13a3a-2bfc-4c01-a231-778e8696b553 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:31.543972 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-46a13a3a-2bfc-4c01-a231-778e8696b553 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:31.559334 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a13a3a-2bfc-4c01-a231-778e8696b553 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:31.560968 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a13a3a-2bfc-4c01-a231-778e8696b553 service nova] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:31.573954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1420/2812] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:08:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:08:33.262885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613978ed-74a8-4edd-b96a-17486bb0b44a service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:33.325603 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613978ed-74a8-4edd-b96a-17486bb0b44a service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:33.330102 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f54ba-d4a7-492c-bba8-e56529068cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:33.330729 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f54ba-d4a7-492c-bba8-e56529068cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:33.331011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f54ba-d4a7-492c-bba8-e56529068cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:33.331315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f54ba-d4a7-492c-bba8-e56529068cb3 admin admin] PATH_INFO: `/v3/users/e62c5e6ae98149cf83a6d0d32119e640` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:33.332444 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f54ba-d4a7-492c-bba8-e56529068cb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e62c5e6ae98149cf83a6d0d32119e640)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:33.334364 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f54ba-d4a7-492c-bba8-e56529068cb3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:33.535758 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6d9f54ba-d4a7-492c-bba8-e56529068cb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e62c5e6ae98149cf83a6d0d32119e640'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:33.542746 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6d9f54ba-d4a7-492c-bba8-e56529068cb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e62c5e6ae98149cf83a6d0d32119e640'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:33.544018 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6d9f54ba-d4a7-492c-bba8-e56529068cb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e62c5e6ae98149cf83a6d0d32119e640'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:33.602294 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6d9f54ba-d4a7-492c-bba8-e56529068cb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e62c5e6ae98149cf83a6d0d32119e640'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:33.615874 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-6d9f54ba-d4a7-492c-bba8-e56529068cb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e62c5e6ae98149cf83a6d0d32119e640'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:33.622424 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1393/2813] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:08:33 2022] DELETE /identity/v3/users/e62c5e6ae98149cf83a6d0d32119e640 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:08:33.626907 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d59ac-2643-4421-95e8-9e451308ea6f None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:33.627418 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d59ac-2643-4421-95e8-9e451308ea6f None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:33.627506 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d59ac-2643-4421-95e8-9e451308ea6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:33.648030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-617d59ac-2643-4421-95e8-9e451308ea6f None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:08:33.649368 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-617d59ac-2643-4421-95e8-9e451308ea6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:33.714426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1421/2814] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:08:33.724466 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f54ba-d4a7-492c-bba8-e56529068cb3 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:33.725977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6d9f54ba-d4a7-492c-bba8-e56529068cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:33.783763 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f54ba-d4a7-492c-bba8-e56529068cb3 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:33.787343 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111cbc5-5fc1-451d-be5f-b604e37b9a17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:33.787946 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111cbc5-5fc1-451d-be5f-b604e37b9a17 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:33.788226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111cbc5-5fc1-451d-be5f-b604e37b9a17 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:33.788492 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111cbc5-5fc1-451d-be5f-b604e37b9a17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:33.798184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111cbc5-5fc1-451d-be5f-b604e37b9a17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:33.799732 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111cbc5-5fc1-451d-be5f-b604e37b9a17 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:33.810099 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1394/2815] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:08:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:08:34.241954 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d59ac-2643-4421-95e8-9e451308ea6f None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:34.243373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-617d59ac-2643-4421-95e8-9e451308ea6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:34.255672 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d59ac-2643-4421-95e8-9e451308ea6f None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:34.260290 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c6866-7724-4a85-b6f3-d62d0e1cad04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:34.260928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c6866-7724-4a85-b6f3-d62d0e1cad04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:34.261208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c6866-7724-4a85-b6f3-d62d0e1cad04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:34.261478 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c6866-7724-4a85-b6f3-d62d0e1cad04 admin admin] PATH_INFO: `/v3/projects/fad9a3739a7f4b74bdf62645ff77bae2` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:34.263120 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c6866-7724-4a85-b6f3-d62d0e1cad04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fad9a3739a7f4b74bdf62645ff77bae2)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:34.265080 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c6866-7724-4a85-b6f3-d62d0e1cad04 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:34.366048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c72c6866-7724-4a85-b6f3-d62d0e1cad04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fad9a3739a7f4b74bdf62645ff77bae2', 'request_id': 'req-c72c6866-7724-4a85-b6f3-d62d0e1cad04'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:34.371519 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c72c6866-7724-4a85-b6f3-d62d0e1cad04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fad9a3739a7f4b74bdf62645ff77bae2', 'request_id': 'req-c72c6866-7724-4a85-b6f3-d62d0e1cad04'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:34.440299 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c72c6866-7724-4a85-b6f3-d62d0e1cad04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fad9a3739a7f4b74bdf62645ff77bae2', 'request_id': 'req-c72c6866-7724-4a85-b6f3-d62d0e1cad04'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:34.452060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c72c6866-7724-4a85-b6f3-d62d0e1cad04 admin admin] The token cache is being invalidate because project fad9a3739a7f4b74bdf62645ff77bae2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:08:34.453120 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c72c6866-7724-4a85-b6f3-d62d0e1cad04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:34.453120 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c72c6866-7724-4a85-b6f3-d62d0e1cad04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:08:34.454631 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1422/2816] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:08:34 2022] DELETE /identity/v3/projects/fad9a3739a7f4b74bdf62645ff77bae2 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:08:34.459506 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05c568-0b67-4d02-90a8-ca56bfe3f324 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:34.459986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05c568-0b67-4d02-90a8-ca56bfe3f324 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:34.460107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05c568-0b67-4d02-90a8-ca56bfe3f324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:34.479966 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-9d05c568-0b67-4d02-90a8-ca56bfe3f324 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:08:34.481165 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-9d05c568-0b67-4d02-90a8-ca56bfe3f324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:34.498114 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:08:34.539780 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1395/2817] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:08:34.547638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16049c-19a8-4117-9db7-47dc2d71a21d None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:34.547927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16049c-19a8-4117-9db7-47dc2d71a21d None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:34.548231 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16049c-19a8-4117-9db7-47dc2d71a21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:34.568182 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-cd16049c-19a8-4117-9db7-47dc2d71a21d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:08:34.569389 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-cd16049c-19a8-4117-9db7-47dc2d71a21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:34.578017 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1423/2818] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:08:34.582275 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05c568-0b67-4d02-90a8-ca56bfe3f324 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:34.592710 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05c568-0b67-4d02-90a8-ca56bfe3f324 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:34.597379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477b06f-1554-4512-a2c4-8faa5f7bc4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:34.597976 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477b06f-1554-4512-a2c4-8faa5f7bc4de admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:34.598256 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477b06f-1554-4512-a2c4-8faa5f7bc4de admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:34.598531 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477b06f-1554-4512-a2c4-8faa5f7bc4de admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:34.599222 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477b06f-1554-4512-a2c4-8faa5f7bc4de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:08:34.599528 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477b06f-1554-4512-a2c4-8faa5f7bc4de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:34.601145 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477b06f-1554-4512-a2c4-8faa5f7bc4de admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:34.612538 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1396/2819] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:08:34.616785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16049c-19a8-4117-9db7-47dc2d71a21d None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:34.626536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16049c-19a8-4117-9db7-47dc2d71a21d None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:34.631405 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd0c0d-f18a-4720-8126-fccd11e66b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:34.632020 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0c0d-f18a-4720-8126-fccd11e66b9a admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:34.632264 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0c0d-f18a-4720-8126-fccd11e66b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:34.632538 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0c0d-f18a-4720-8126-fccd11e66b9a admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:34.633208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd0c0d-f18a-4720-8126-fccd11e66b9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:34.635300 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd0c0d-f18a-4720-8126-fccd11e66b9a admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:34.723394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1424/2820] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:08:34 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:08:34.727849 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477b06f-1554-4512-a2c4-8faa5f7bc4de admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:34.737883 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477b06f-1554-4512-a2c4-8faa5f7bc4de admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:34.793151 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e593f8-33d1-4b68-87eb-e28375aa1468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:34.793751 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e593f8-33d1-4b68-87eb-e28375aa1468 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:34.794031 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e593f8-33d1-4b68-87eb-e28375aa1468 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:34.794306 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e593f8-33d1-4b68-87eb-e28375aa1468 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:34.794948 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e593f8-33d1-4b68-87eb-e28375aa1468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:34.796894 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e593f8-33d1-4b68-87eb-e28375aa1468 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:34.881632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1397/2821] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:08:34 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:08:34.888044 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd0c0d-f18a-4720-8126-fccd11e66b9a admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:34.898413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd0c0d-f18a-4720-8126-fccd11e66b9a admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:34.903147 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8f033-73b7-4720-8305-357ab0688bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:34.903734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8f033-73b7-4720-8305-357ab0688bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:34.904012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8f033-73b7-4720-8305-357ab0688bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:34.904282 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8f033-73b7-4720-8305-357ab0688bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:34.904919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8f033-73b7-4720-8305-357ab0688bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:34.906510 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8f033-73b7-4720-8305-357ab0688bf9 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:34.916457 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1425/2822] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:08:34.920404 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e593f8-33d1-4b68-87eb-e28375aa1468 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:34.930297 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e593f8-33d1-4b68-87eb-e28375aa1468 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:34.935376 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef41da-911e-4c01-a57e-166019dd4741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:34.935958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef41da-911e-4c01-a57e-166019dd4741 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:34.936229 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef41da-911e-4c01-a57e-166019dd4741 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:34.936493 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef41da-911e-4c01-a57e-166019dd4741 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:34.937123 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef41da-911e-4c01-a57e-166019dd4741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:34.938700 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef41da-911e-4c01-a57e-166019dd4741 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:34.949861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1398/2823] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:08:34.953801 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8f033-73b7-4720-8305-357ab0688bf9 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:34.963585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8f033-73b7-4720-8305-357ab0688bf9 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:34.968454 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718eacae-c0c1-483a-a9b0-aa624ec69c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:34.969078 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718eacae-c0c1-483a-a9b0-aa624ec69c6c admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:34.969357 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718eacae-c0c1-483a-a9b0-aa624ec69c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:34.969633 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718eacae-c0c1-483a-a9b0-aa624ec69c6c admin admin] PATH_INFO: `/v3/projects/2bb779c097914973a01e24b8490c5495/users/53c59a745dc94a8299d10c513436a894/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:34.984376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718eacae-c0c1-483a-a9b0-aa624ec69c6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb779c097914973a01e24b8490c5495, user_id=53c59a745dc94a8299d10c513436a894, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:34.987036 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718eacae-c0c1-483a-a9b0-aa624ec69c6c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:35.088858 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1426/2824] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:08:34 2022] PUT /identity/v3/projects/2bb779c097914973a01e24b8490c5495/users/53c59a745dc94a8299d10c513436a894/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:08:35.099234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08cd56-9041-42ff-a90c-6653504534b5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:35.099517 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08cd56-9041-42ff-a90c-6653504534b5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:35.099792 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08cd56-9041-42ff-a90c-6653504534b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:35.128886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-bb08cd56-9041-42ff-a90c-6653504534b5 None None] MFA Rules not processed for user `53c59a745dc94a8299d10c513436a894`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:08:35.130101 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-bb08cd56-9041-42ff-a90c-6653504534b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:35.197011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1399/2825] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:08:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:08:35.201489 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718eacae-c0c1-483a-a9b0-aa624ec69c6c admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:35.206056 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:08:35.267681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718eacae-c0c1-483a-a9b0-aa624ec69c6c admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:35.272370 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b51c5-adb8-44f9-9479-4a652e5b6075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:35.272967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b51c5-adb8-44f9-9479-4a652e5b6075 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:35.273243 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b51c5-adb8-44f9-9479-4a652e5b6075 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:35.273519 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b51c5-adb8-44f9-9479-4a652e5b6075 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:35.274139 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b51c5-adb8-44f9-9479-4a652e5b6075 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:35.276090 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b51c5-adb8-44f9-9479-4a652e5b6075 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:35.368985 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1427/2826] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:08:35 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:08:35.373176 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08cd56-9041-42ff-a90c-6653504534b5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:35.383886 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08cd56-9041-42ff-a90c-6653504534b5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:35.395791 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:08:35.439995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f631de-a4e8-4335-bfdf-a08c2c7ffe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:35.440666 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f631de-a4e8-4335-bfdf-a08c2c7ffe32 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:35.440948 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f631de-a4e8-4335-bfdf-a08c2c7ffe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:35.441223 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f631de-a4e8-4335-bfdf-a08c2c7ffe32 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:35.441859 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f631de-a4e8-4335-bfdf-a08c2c7ffe32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:35.443802 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f631de-a4e8-4335-bfdf-a08c2c7ffe32 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:35.528111 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1400/2827] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:08:35 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:08:35.532256 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b51c5-adb8-44f9-9479-4a652e5b6075 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:35.542434 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b51c5-adb8-44f9-9479-4a652e5b6075 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:35.547702 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68633c39-56dd-42ef-89e2-f90be251f85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:35.548285 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68633c39-56dd-42ef-89e2-f90be251f85e admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:35.548563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68633c39-56dd-42ef-89e2-f90be251f85e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:35.548834 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68633c39-56dd-42ef-89e2-f90be251f85e admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:35.549474 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68633c39-56dd-42ef-89e2-f90be251f85e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:35.551059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68633c39-56dd-42ef-89e2-f90be251f85e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:35.561536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1428/2828] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:08:35.565500 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f631de-a4e8-4335-bfdf-a08c2c7ffe32 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:35.575353 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f631de-a4e8-4335-bfdf-a08c2c7ffe32 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:35.579873 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9c591-9fbe-4946-810e-ba0f4d38ef17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:35.580501 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9c591-9fbe-4946-810e-ba0f4d38ef17 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:35.580774 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9c591-9fbe-4946-810e-ba0f4d38ef17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:35.581041 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9c591-9fbe-4946-810e-ba0f4d38ef17 admin admin] PATH_INFO: `/v3/projects/65f1b28f0bbe42b4aff77c2b21363d65/users/9e6c26c1fdd542c98057e6eb9eb37630/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:35.596525 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9c591-9fbe-4946-810e-ba0f4d38ef17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f1b28f0bbe42b4aff77c2b21363d65, user_id=9e6c26c1fdd542c98057e6eb9eb37630, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:35.599201 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9c591-9fbe-4946-810e-ba0f4d38ef17 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:35.669097 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1401/2829] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:08:35 2022] PUT /identity/v3/projects/65f1b28f0bbe42b4aff77c2b21363d65/users/9e6c26c1fdd542c98057e6eb9eb37630/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:08:35.680366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68633c39-56dd-42ef-89e2-f90be251f85e admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:35.690524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68633c39-56dd-42ef-89e2-f90be251f85e admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:35.745727 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f7d8b-1766-4053-bd2f-4c44bed2f146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:35.746354 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f7d8b-1766-4053-bd2f-4c44bed2f146 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:35.746632 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f7d8b-1766-4053-bd2f-4c44bed2f146 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:35.746903 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f7d8b-1766-4053-bd2f-4c44bed2f146 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:35.747569 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f7d8b-1766-4053-bd2f-4c44bed2f146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:35.749183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f7d8b-1766-4053-bd2f-4c44bed2f146 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:35.758643 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1429/2830] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:08:35.763332 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9c591-9fbe-4946-810e-ba0f4d38ef17 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:35.773585 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9c591-9fbe-4946-810e-ba0f4d38ef17 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:35.778860 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a8432-e77e-4b98-8a15-55940e26cb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:35.779531 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a8432-e77e-4b98-8a15-55940e26cb79 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:35.779804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a8432-e77e-4b98-8a15-55940e26cb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:35.780074 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a8432-e77e-4b98-8a15-55940e26cb79 admin admin] PATH_INFO: `/v3/projects/65f1b28f0bbe42b4aff77c2b21363d65/users/9e6c26c1fdd542c98057e6eb9eb37630/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:35.781983 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a8432-e77e-4b98-8a15-55940e26cb79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f1b28f0bbe42b4aff77c2b21363d65, user_id=9e6c26c1fdd542c98057e6eb9eb37630, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:35.784671 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a8432-e77e-4b98-8a15-55940e26cb79 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:35.794484 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1402/2831] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:08:35 2022] PUT /identity/v3/projects/65f1b28f0bbe42b4aff77c2b21363d65/users/9e6c26c1fdd542c98057e6eb9eb37630/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:08:35.804661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5cd90-eb5a-441e-a673-65ed3ee40bb8 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:35.804943 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5cd90-eb5a-441e-a673-65ed3ee40bb8 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:35.805214 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5cd90-eb5a-441e-a673-65ed3ee40bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:35.833937 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-63b5cd90-eb5a-441e-a673-65ed3ee40bb8 None None] MFA Rules not processed for user `9e6c26c1fdd542c98057e6eb9eb37630`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:08:35.835182 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-63b5cd90-eb5a-441e-a673-65ed3ee40bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:35.897268 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:08:35.903060 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1430/2832] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:08:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:08:35.914345 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a8432-e77e-4b98-8a15-55940e26cb79 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:35.916071 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-f70a8432-e77e-4b98-8a15-55940e26cb79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:35.974074 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a8432-e77e-4b98-8a15-55940e26cb79 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:35.977748 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307462d6-9125-4eee-af84-de8f2014f9eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:35.978340 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307462d6-9125-4eee-af84-de8f2014f9eb service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:35.978620 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307462d6-9125-4eee-af84-de8f2014f9eb service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:35.978892 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307462d6-9125-4eee-af84-de8f2014f9eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:35.987904 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307462d6-9125-4eee-af84-de8f2014f9eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:35.989415 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307462d6-9125-4eee-af84-de8f2014f9eb service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:35.998989 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1403/2833] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:08:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:08:36.180456 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5cd90-eb5a-441e-a673-65ed3ee40bb8 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:36.190606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5cd90-eb5a-441e-a673-65ed3ee40bb8 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:36.194118 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ae264-526c-443a-91d6-f9f88bdae78d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:36.194703 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ae264-526c-443a-91d6-f9f88bdae78d service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:36.194970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ae264-526c-443a-91d6-f9f88bdae78d service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:36.195267 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ae264-526c-443a-91d6-f9f88bdae78d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:36.251681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ae264-526c-443a-91d6-f9f88bdae78d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:36.253135 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ae264-526c-443a-91d6-f9f88bdae78d service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:36.266015 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1431/2834] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:08:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:08:45.994580 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:08:46.258275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=29 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:20.883577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307462d6-9125-4eee-af84-de8f2014f9eb service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:20.944556 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307462d6-9125-4eee-af84-de8f2014f9eb service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:20.949039 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f166eb-990e-4da9-9bc8-b07407129ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:20.949658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f166eb-990e-4da9-9bc8-b07407129ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:20.949937 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f166eb-990e-4da9-9bc8-b07407129ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:20.950211 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f166eb-990e-4da9-9bc8-b07407129ce4 admin admin] PATH_INFO: `/v3/users/53c59a745dc94a8299d10c513436a894` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:20.951421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f166eb-990e-4da9-9bc8-b07407129ce4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53c59a745dc94a8299d10c513436a894)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:20.954321 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f166eb-990e-4da9-9bc8-b07407129ce4 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:21.111329 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c6f166eb-990e-4da9-9bc8-b07407129ce4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53c59a745dc94a8299d10c513436a894'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:21.118565 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c6f166eb-990e-4da9-9bc8-b07407129ce4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53c59a745dc94a8299d10c513436a894'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:21.119200 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c6f166eb-990e-4da9-9bc8-b07407129ce4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53c59a745dc94a8299d10c513436a894'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:21.182686 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c6f166eb-990e-4da9-9bc8-b07407129ce4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53c59a745dc94a8299d10c513436a894'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:21.194858 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-c6f166eb-990e-4da9-9bc8-b07407129ce4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53c59a745dc94a8299d10c513436a894'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:21.201045 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1404/2835] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:09:20 2022] DELETE /identity/v3/users/53c59a745dc94a8299d10c513436a894 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:09:21.205750 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ad822-adc7-49f8-8974-b0e84a6ac3dc None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:21.206309 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ad822-adc7-49f8-8974-b0e84a6ac3dc None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:21.206583 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ad822-adc7-49f8-8974-b0e84a6ac3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:21.227155 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-ac0ad822-adc7-49f8-8974-b0e84a6ac3dc None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:21.228344 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ac0ad822-adc7-49f8-8974-b0e84a6ac3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:21.286515 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1432/2836] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:09:21.297117 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f166eb-990e-4da9-9bc8-b07407129ce4 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:21.298547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c6f166eb-990e-4da9-9bc8-b07407129ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:21.353355 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f166eb-990e-4da9-9bc8-b07407129ce4 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:21.356926 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29165f80-06ad-48c1-b0c1-6756ed832675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:21.358174 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29165f80-06ad-48c1-b0c1-6756ed832675 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:21.358447 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29165f80-06ad-48c1-b0c1-6756ed832675 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:21.358712 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29165f80-06ad-48c1-b0c1-6756ed832675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:21.367547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29165f80-06ad-48c1-b0c1-6756ed832675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:21.369008 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29165f80-06ad-48c1-b0c1-6756ed832675 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:21.378295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1405/2837] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:09:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:09:21.830736 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ad822-adc7-49f8-8974-b0e84a6ac3dc None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:21.832197 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ac0ad822-adc7-49f8-8974-b0e84a6ac3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:21.844393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ad822-adc7-49f8-8974-b0e84a6ac3dc None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:21.849326 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b1ccf-17d8-48cc-816f-9082b0341c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:21.849986 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b1ccf-17d8-48cc-816f-9082b0341c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:21.850923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b1ccf-17d8-48cc-816f-9082b0341c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:21.850923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b1ccf-17d8-48cc-816f-9082b0341c86 admin admin] PATH_INFO: `/v3/projects/2bb779c097914973a01e24b8490c5495` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:21.851960 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b1ccf-17d8-48cc-816f-9082b0341c86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bb779c097914973a01e24b8490c5495)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:21.853976 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b1ccf-17d8-48cc-816f-9082b0341c86 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:21.960222 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-403b1ccf-17d8-48cc-816f-9082b0341c86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bb779c097914973a01e24b8490c5495', 'request_id': 'req-403b1ccf-17d8-48cc-816f-9082b0341c86'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:21.965891 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-403b1ccf-17d8-48cc-816f-9082b0341c86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bb779c097914973a01e24b8490c5495', 'request_id': 'req-403b1ccf-17d8-48cc-816f-9082b0341c86'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:22.045935 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-403b1ccf-17d8-48cc-816f-9082b0341c86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bb779c097914973a01e24b8490c5495', 'request_id': 'req-403b1ccf-17d8-48cc-816f-9082b0341c86'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:22.057044 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-403b1ccf-17d8-48cc-816f-9082b0341c86 admin admin] The token cache is being invalidate because project 2bb779c097914973a01e24b8490c5495 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:09:22.062704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-403b1ccf-17d8-48cc-816f-9082b0341c86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:22.062704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-403b1ccf-17d8-48cc-816f-9082b0341c86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:22.062704 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1433/2838] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:09:21 2022] DELETE /identity/v3/projects/2bb779c097914973a01e24b8490c5495 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:09:22.064146 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29165f80-06ad-48c1-b0c1-6756ed832675 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:22.073728 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29165f80-06ad-48c1-b0c1-6756ed832675 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:22.092315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:22.129948 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22ceb2-631c-452c-9b3b-6fbb7eb8a8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:22.130592 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22ceb2-631c-452c-9b3b-6fbb7eb8a8d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:22.130961 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22ceb2-631c-452c-9b3b-6fbb7eb8a8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:22.131265 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22ceb2-631c-452c-9b3b-6fbb7eb8a8d3 admin admin] PATH_INFO: `/v3/users/9e6c26c1fdd542c98057e6eb9eb37630` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:22.132380 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22ceb2-631c-452c-9b3b-6fbb7eb8a8d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e6c26c1fdd542c98057e6eb9eb37630)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:22.134300 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22ceb2-631c-452c-9b3b-6fbb7eb8a8d3 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:22.264635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-dc22ceb2-631c-452c-9b3b-6fbb7eb8a8d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e6c26c1fdd542c98057e6eb9eb37630'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:22.272121 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-dc22ceb2-631c-452c-9b3b-6fbb7eb8a8d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e6c26c1fdd542c98057e6eb9eb37630'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:22.272846 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-dc22ceb2-631c-452c-9b3b-6fbb7eb8a8d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e6c26c1fdd542c98057e6eb9eb37630'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:22.341824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-dc22ceb2-631c-452c-9b3b-6fbb7eb8a8d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e6c26c1fdd542c98057e6eb9eb37630'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:22.353836 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-dc22ceb2-631c-452c-9b3b-6fbb7eb8a8d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e6c26c1fdd542c98057e6eb9eb37630'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:22.360245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1406/2839] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:09:22 2022] DELETE /identity/v3/users/9e6c26c1fdd542c98057e6eb9eb37630 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:09:22.818499 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b1ccf-17d8-48cc-816f-9082b0341c86 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:22.819971 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-403b1ccf-17d8-48cc-816f-9082b0341c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:22.833185 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b1ccf-17d8-48cc-816f-9082b0341c86 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:22.891928 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266a49d-487b-477e-9999-17909cad48ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:22.892556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266a49d-487b-477e-9999-17909cad48ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:22.892840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266a49d-487b-477e-9999-17909cad48ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:22.893112 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266a49d-487b-477e-9999-17909cad48ed admin admin] PATH_INFO: `/v3/projects/65f1b28f0bbe42b4aff77c2b21363d65` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:22.894247 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266a49d-487b-477e-9999-17909cad48ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=65f1b28f0bbe42b4aff77c2b21363d65)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:22.896268 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266a49d-487b-477e-9999-17909cad48ed admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:22.995979 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:23.012134 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f266a49d-487b-477e-9999-17909cad48ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65f1b28f0bbe42b4aff77c2b21363d65', 'request_id': 'req-f266a49d-487b-477e-9999-17909cad48ed'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:23.018078 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f266a49d-487b-477e-9999-17909cad48ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65f1b28f0bbe42b4aff77c2b21363d65', 'request_id': 'req-f266a49d-487b-477e-9999-17909cad48ed'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:23.092267 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f266a49d-487b-477e-9999-17909cad48ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65f1b28f0bbe42b4aff77c2b21363d65', 'request_id': 'req-f266a49d-487b-477e-9999-17909cad48ed'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:23.103686 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f266a49d-487b-477e-9999-17909cad48ed admin admin] The token cache is being invalidate because project 65f1b28f0bbe42b4aff77c2b21363d65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:09:23.103973 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f266a49d-487b-477e-9999-17909cad48ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:23.104602 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-f266a49d-487b-477e-9999-17909cad48ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:23.106067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1434/2840] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:09:22 2022] DELETE /identity/v3/projects/65f1b28f0bbe42b4aff77c2b21363d65 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:09:23.110991 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a81312-da3a-4545-aa8a-1c0179de31f3 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:23.111503 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a81312-da3a-4545-aa8a-1c0179de31f3 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:23.111611 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a81312-da3a-4545-aa8a-1c0179de31f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:23.130817 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-81a81312-da3a-4545-aa8a-1c0179de31f3 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:23.132034 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-81a81312-da3a-4545-aa8a-1c0179de31f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:23.189908 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1407/2841] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:09:23.197674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e986f14-490b-4bed-8c33-4c0761932046 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:23.197957 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e986f14-490b-4bed-8c33-4c0761932046 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:23.198236 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e986f14-490b-4bed-8c33-4c0761932046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:23.218642 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-0e986f14-490b-4bed-8c33-4c0761932046 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:23.219874 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-0e986f14-490b-4bed-8c33-4c0761932046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:23.227646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1435/2842] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:09:23.231636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a81312-da3a-4545-aa8a-1c0179de31f3 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:23.241391 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a81312-da3a-4545-aa8a-1c0179de31f3 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:23.246036 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdbc35-071c-481e-bcf9-a4016aab3c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:23.246631 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdbc35-071c-481e-bcf9-a4016aab3c5a admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:23.246912 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdbc35-071c-481e-bcf9-a4016aab3c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:23.247219 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdbc35-071c-481e-bcf9-a4016aab3c5a admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:23.248359 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fdbc35-071c-481e-bcf9-a4016aab3c5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:09:23.248681 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fdbc35-071c-481e-bcf9-a4016aab3c5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:23.250295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fdbc35-071c-481e-bcf9-a4016aab3c5a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:23.261664 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1408/2843] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:09:23.266528 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e986f14-490b-4bed-8c33-4c0761932046 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:23.275791 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e986f14-490b-4bed-8c33-4c0761932046 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:23.280417 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64e135-dc60-4776-91ec-ab3914d72420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:23.281002 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64e135-dc60-4776-91ec-ab3914d72420 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:23.281281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64e135-dc60-4776-91ec-ab3914d72420 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:23.281548 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64e135-dc60-4776-91ec-ab3914d72420 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:23.282180 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64e135-dc60-4776-91ec-ab3914d72420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:23.284146 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64e135-dc60-4776-91ec-ab3914d72420 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:23.367661 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1436/2844] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:09:23 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:09:23.371724 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdbc35-071c-481e-bcf9-a4016aab3c5a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:23.382176 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdbc35-071c-481e-bcf9-a4016aab3c5a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:23.409410 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:23.439239 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe08d8-acd4-4fec-be0a-f906e40a5772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:23.439963 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe08d8-acd4-4fec-be0a-f906e40a5772 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:23.440287 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe08d8-acd4-4fec-be0a-f906e40a5772 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:23.440917 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe08d8-acd4-4fec-be0a-f906e40a5772 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:23.441568 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe08d8-acd4-4fec-be0a-f906e40a5772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:23.443515 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe08d8-acd4-4fec-be0a-f906e40a5772 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:23.508928 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1409/2845] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:09:23 2022] POST /identity/v3/users => generated 282 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:09:23.513038 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64e135-dc60-4776-91ec-ab3914d72420 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:23.524261 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64e135-dc60-4776-91ec-ab3914d72420 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:23.529086 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f256a48-c68e-4220-999c-7264ab78aba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:23.529674 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f256a48-c68e-4220-999c-7264ab78aba4 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:23.529952 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f256a48-c68e-4220-999c-7264ab78aba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:23.530329 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f256a48-c68e-4220-999c-7264ab78aba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:23.531001 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f256a48-c68e-4220-999c-7264ab78aba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:23.532631 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f256a48-c68e-4220-999c-7264ab78aba4 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:23.542520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1437/2846] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:09:23.546394 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe08d8-acd4-4fec-be0a-f906e40a5772 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:23.556496 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe08d8-acd4-4fec-be0a-f906e40a5772 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:23.561623 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c50604-f5e2-424e-8d72-537ef863375e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:23.562203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c50604-f5e2-424e-8d72-537ef863375e admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:23.562477 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c50604-f5e2-424e-8d72-537ef863375e admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:23.562744 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c50604-f5e2-424e-8d72-537ef863375e admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:23.563412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c50604-f5e2-424e-8d72-537ef863375e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:23.564987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c50604-f5e2-424e-8d72-537ef863375e admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:23.575378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1410/2847] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:09:23.580127 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f256a48-c68e-4220-999c-7264ab78aba4 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:23.590048 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f256a48-c68e-4220-999c-7264ab78aba4 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:23.594827 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac600a2b-4348-4ee8-92b2-e8ad3df0ffb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:23.595501 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac600a2b-4348-4ee8-92b2-e8ad3df0ffb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:23.595769 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac600a2b-4348-4ee8-92b2-e8ad3df0ffb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:23.596034 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac600a2b-4348-4ee8-92b2-e8ad3df0ffb5 admin admin] PATH_INFO: `/v3/projects/c59369dec1a744b98d82b6b55d0e2e0f/users/5b3c7a27dd2c4a6c849905ee9516ecb7/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:23.611497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac600a2b-4348-4ee8-92b2-e8ad3df0ffb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c59369dec1a744b98d82b6b55d0e2e0f, user_id=5b3c7a27dd2c4a6c849905ee9516ecb7, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:23.614172 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac600a2b-4348-4ee8-92b2-e8ad3df0ffb5 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:23.685903 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1438/2848] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:09:23 2022] PUT /identity/v3/projects/c59369dec1a744b98d82b6b55d0e2e0f/users/5b3c7a27dd2c4a6c849905ee9516ecb7/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:09:23.696818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd10cf-d62c-4fbc-8d3a-9738e41a1deb None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:23.697097 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd10cf-d62c-4fbc-8d3a-9738e41a1deb None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:23.698997 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd10cf-d62c-4fbc-8d3a-9738e41a1deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:23.724707 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-80cd10cf-d62c-4fbc-8d3a-9738e41a1deb None None] MFA Rules not processed for user `5b3c7a27dd2c4a6c849905ee9516ecb7`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:23.725921 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-80cd10cf-d62c-4fbc-8d3a-9738e41a1deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:23.789094 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1411/2849] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:09:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:09:23.793675 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac600a2b-4348-4ee8-92b2-e8ad3df0ffb5 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:23.855627 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac600a2b-4348-4ee8-92b2-e8ad3df0ffb5 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:23.861064 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159cc9b-07b9-43b6-bae7-f83aecbd6f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:23.861693 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159cc9b-07b9-43b6-bae7-f83aecbd6f8c admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:23.861970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159cc9b-07b9-43b6-bae7-f83aecbd6f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:23.862240 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159cc9b-07b9-43b6-bae7-f83aecbd6f8c admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:23.862901 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159cc9b-07b9-43b6-bae7-f83aecbd6f8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:23.864874 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159cc9b-07b9-43b6-bae7-f83aecbd6f8c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:23.927113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1439/2850] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:09:23 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:09:23.931359 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd10cf-d62c-4fbc-8d3a-9738e41a1deb None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:23.942697 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd10cf-d62c-4fbc-8d3a-9738e41a1deb None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:23.997261 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58839293-d705-406d-bbbb-37e2d2773412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:23.997851 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58839293-d705-406d-bbbb-37e2d2773412 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:23.998126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58839293-d705-406d-bbbb-37e2d2773412 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:23.998395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58839293-d705-406d-bbbb-37e2d2773412 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:23.999045 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58839293-d705-406d-bbbb-37e2d2773412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:24.000960 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58839293-d705-406d-bbbb-37e2d2773412 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:24.131776 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1412/2851] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:09:23 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:09:24.135790 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159cc9b-07b9-43b6-bae7-f83aecbd6f8c admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:24.145995 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159cc9b-07b9-43b6-bae7-f83aecbd6f8c admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:24.150502 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829c5b8-e947-4dd8-9dfd-5f4080c9fd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:24.151089 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829c5b8-e947-4dd8-9dfd-5f4080c9fd0c admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:24.151460 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829c5b8-e947-4dd8-9dfd-5f4080c9fd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:24.151731 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829c5b8-e947-4dd8-9dfd-5f4080c9fd0c admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:24.152366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829c5b8-e947-4dd8-9dfd-5f4080c9fd0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:24.153959 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829c5b8-e947-4dd8-9dfd-5f4080c9fd0c admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:24.164903 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1440/2852] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:09:24.169075 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58839293-d705-406d-bbbb-37e2d2773412 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:24.179494 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58839293-d705-406d-bbbb-37e2d2773412 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:24.184105 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee72a3-a59a-4641-8912-900c567fb0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:24.184741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee72a3-a59a-4641-8912-900c567fb0fd admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:24.185018 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee72a3-a59a-4641-8912-900c567fb0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:24.185275 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee72a3-a59a-4641-8912-900c567fb0fd admin admin] PATH_INFO: `/v3/projects/399b4b2b4e5f42f2a517ac330bd8e58b/users/2c0627fc5bfb4f509fa1e95f14376e90/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:24.199683 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee72a3-a59a-4641-8912-900c567fb0fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=399b4b2b4e5f42f2a517ac330bd8e58b, user_id=2c0627fc5bfb4f509fa1e95f14376e90, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:24.202245 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee72a3-a59a-4641-8912-900c567fb0fd admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:24.268284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1413/2853] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:09:24 2022] PUT /identity/v3/projects/399b4b2b4e5f42f2a517ac330bd8e58b/users/2c0627fc5bfb4f509fa1e95f14376e90/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:09:24.272358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829c5b8-e947-4dd8-9dfd-5f4080c9fd0c admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:24.282815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829c5b8-e947-4dd8-9dfd-5f4080c9fd0c admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:24.302585 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:24.336983 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c7c41-7277-47d4-840b-4e6658cedf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:24.337568 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c7c41-7277-47d4-840b-4e6658cedf8b admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:24.337843 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c7c41-7277-47d4-840b-4e6658cedf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:24.338112 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c7c41-7277-47d4-840b-4e6658cedf8b admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:24.338736 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c7c41-7277-47d4-840b-4e6658cedf8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:24.340598 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c7c41-7277-47d4-840b-4e6658cedf8b admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:24.351388 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1441/2854] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:09:24.355632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee72a3-a59a-4641-8912-900c567fb0fd admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:24.361772 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:24.367121 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee72a3-a59a-4641-8912-900c567fb0fd admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:24.371861 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175be76a-b8c2-4245-93ba-dc28740a3fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:24.372495 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175be76a-b8c2-4245-93ba-dc28740a3fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:24.372770 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175be76a-b8c2-4245-93ba-dc28740a3fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:24.373039 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175be76a-b8c2-4245-93ba-dc28740a3fb9 admin admin] PATH_INFO: `/v3/projects/399b4b2b4e5f42f2a517ac330bd8e58b/users/2c0627fc5bfb4f509fa1e95f14376e90/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:24.374947 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175be76a-b8c2-4245-93ba-dc28740a3fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=399b4b2b4e5f42f2a517ac330bd8e58b, user_id=2c0627fc5bfb4f509fa1e95f14376e90, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:24.377710 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175be76a-b8c2-4245-93ba-dc28740a3fb9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:24.388039 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1414/2855] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:09:24 2022] PUT /identity/v3/projects/399b4b2b4e5f42f2a517ac330bd8e58b/users/2c0627fc5bfb4f509fa1e95f14376e90/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:09:24.453006 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c2d14-c0f1-449e-afd4-8807531b15f6 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:24.453258 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c2d14-c0f1-449e-afd4-8807531b15f6 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:24.453559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c2d14-c0f1-449e-afd4-8807531b15f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:24.482376 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-a38c2d14-c0f1-449e-afd4-8807531b15f6 None None] MFA Rules not processed for user `2c0627fc5bfb4f509fa1e95f14376e90`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:24.483647 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-a38c2d14-c0f1-449e-afd4-8807531b15f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:24.550614 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1442/2856] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:09:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:09:24.561829 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175be76a-b8c2-4245-93ba-dc28740a3fb9 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:24.563589 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-175be76a-b8c2-4245-93ba-dc28740a3fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:24.619955 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175be76a-b8c2-4245-93ba-dc28740a3fb9 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:24.623803 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f552d-11ef-40e6-89a5-c20f60c58a28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:24.624438 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f552d-11ef-40e6-89a5-c20f60c58a28 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:24.624717 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f552d-11ef-40e6-89a5-c20f60c58a28 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:24.624987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f552d-11ef-40e6-89a5-c20f60c58a28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:24.634385 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f552d-11ef-40e6-89a5-c20f60c58a28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:24.635923 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f552d-11ef-40e6-89a5-c20f60c58a28 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:24.645673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1415/2857] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:09:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:09:24.853550 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c2d14-c0f1-449e-afd4-8807531b15f6 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:24.863252 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c2d14-c0f1-449e-afd4-8807531b15f6 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:24.867153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620949d8-bf2c-4386-9989-87946531e9e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:24.867783 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620949d8-bf2c-4386-9989-87946531e9e6 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:24.868059 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620949d8-bf2c-4386-9989-87946531e9e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:24.868357 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620949d8-bf2c-4386-9989-87946531e9e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:24.924193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620949d8-bf2c-4386-9989-87946531e9e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:24.925632 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620949d8-bf2c-4386-9989-87946531e9e6 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:24.934333 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1443/2858] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:09:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:09:28.386827 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f552d-11ef-40e6-89a5-c20f60c58a28 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:28.446987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f552d-11ef-40e6-89a5-c20f60c58a28 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:28.451523 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea6995-29e0-492c-8e3d-ef92e6de8f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:28.452132 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea6995-29e0-492c-8e3d-ef92e6de8f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:28.452409 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea6995-29e0-492c-8e3d-ef92e6de8f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:28.452679 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea6995-29e0-492c-8e3d-ef92e6de8f08 admin admin] PATH_INFO: `/v3/users/5b3c7a27dd2c4a6c849905ee9516ecb7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:28.453794 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea6995-29e0-492c-8e3d-ef92e6de8f08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b3c7a27dd2c4a6c849905ee9516ecb7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:28.455749 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea6995-29e0-492c-8e3d-ef92e6de8f08 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:28.642266 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7aea6995-29e0-492c-8e3d-ef92e6de8f08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b3c7a27dd2c4a6c849905ee9516ecb7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:28.649154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7aea6995-29e0-492c-8e3d-ef92e6de8f08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b3c7a27dd2c4a6c849905ee9516ecb7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:28.649872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7aea6995-29e0-492c-8e3d-ef92e6de8f08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b3c7a27dd2c4a6c849905ee9516ecb7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:28.702766 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7aea6995-29e0-492c-8e3d-ef92e6de8f08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b3c7a27dd2c4a6c849905ee9516ecb7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:28.715402 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-7aea6995-29e0-492c-8e3d-ef92e6de8f08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b3c7a27dd2c4a6c849905ee9516ecb7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:28.721894 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1416/2859] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:09:28 2022] DELETE /identity/v3/users/5b3c7a27dd2c4a6c849905ee9516ecb7 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:09:28.726372 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80948b2-c7c2-4d70-8b5d-ccac54500703 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:28.726825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80948b2-c7c2-4d70-8b5d-ccac54500703 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:28.726925 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80948b2-c7c2-4d70-8b5d-ccac54500703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:28.746503 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-c80948b2-c7c2-4d70-8b5d-ccac54500703 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:28.747735 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c80948b2-c7c2-4d70-8b5d-ccac54500703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:28.787226 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=100 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:28.805969 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1444/2860] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:09:28.815714 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea6995-29e0-492c-8e3d-ef92e6de8f08 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:28.817116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-7aea6995-29e0-492c-8e3d-ef92e6de8f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:28.830299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:28.874339 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea6995-29e0-492c-8e3d-ef92e6de8f08 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:28.877951 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b467b-8b47-4e9a-9dbe-57cdbbd6db29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:28.878547 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b467b-8b47-4e9a-9dbe-57cdbbd6db29 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:28.878841 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b467b-8b47-4e9a-9dbe-57cdbbd6db29 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:28.879118 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b467b-8b47-4e9a-9dbe-57cdbbd6db29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:28.887943 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b467b-8b47-4e9a-9dbe-57cdbbd6db29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:28.889885 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b467b-8b47-4e9a-9dbe-57cdbbd6db29 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:28.899924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1417/2861] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:09:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:09:29.304362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80948b2-c7c2-4d70-8b5d-ccac54500703 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:29.305796 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c80948b2-c7c2-4d70-8b5d-ccac54500703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:29.318433 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80948b2-c7c2-4d70-8b5d-ccac54500703 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:29.323083 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ade590-3fee-4e4e-918d-27e56b89fee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:29.323730 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ade590-3fee-4e4e-918d-27e56b89fee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:29.324005 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ade590-3fee-4e4e-918d-27e56b89fee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:29.324307 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ade590-3fee-4e4e-918d-27e56b89fee7 admin admin] PATH_INFO: `/v3/projects/c59369dec1a744b98d82b6b55d0e2e0f` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:29.325406 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ade590-3fee-4e4e-918d-27e56b89fee7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c59369dec1a744b98d82b6b55d0e2e0f)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:29.327911 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ade590-3fee-4e4e-918d-27e56b89fee7 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:29.428199 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b3ade590-3fee-4e4e-918d-27e56b89fee7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c59369dec1a744b98d82b6b55d0e2e0f', 'request_id': 'req-b3ade590-3fee-4e4e-918d-27e56b89fee7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:29.433800 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b3ade590-3fee-4e4e-918d-27e56b89fee7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c59369dec1a744b98d82b6b55d0e2e0f', 'request_id': 'req-b3ade590-3fee-4e4e-918d-27e56b89fee7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:29.508140 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b3ade590-3fee-4e4e-918d-27e56b89fee7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c59369dec1a744b98d82b6b55d0e2e0f', 'request_id': 'req-b3ade590-3fee-4e4e-918d-27e56b89fee7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:29.519764 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b3ade590-3fee-4e4e-918d-27e56b89fee7 admin admin] The token cache is being invalidate because project c59369dec1a744b98d82b6b55d0e2e0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:09:29.520049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b3ade590-3fee-4e4e-918d-27e56b89fee7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:29.520679 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-b3ade590-3fee-4e4e-918d-27e56b89fee7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:29.522148 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1445/2862] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:09:29 2022] DELETE /identity/v3/projects/c59369dec1a744b98d82b6b55d0e2e0f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:09:29.526150 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b467b-8b47-4e9a-9dbe-57cdbbd6db29 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:29.536061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b467b-8b47-4e9a-9dbe-57cdbbd6db29 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:29.600721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba063c-f6ac-4c48-875e-1eb58042f72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:29.601387 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba063c-f6ac-4c48-875e-1eb58042f72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:29.601694 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba063c-f6ac-4c48-875e-1eb58042f72b admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:29.602022 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba063c-f6ac-4c48-875e-1eb58042f72b admin admin] PATH_INFO: `/v3/users/2c0627fc5bfb4f509fa1e95f14376e90` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:29.603299 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba063c-f6ac-4c48-875e-1eb58042f72b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0627fc5bfb4f509fa1e95f14376e90)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:29.605327 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba063c-f6ac-4c48-875e-1eb58042f72b admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:29.955944 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cfba063c-f6ac-4c48-875e-1eb58042f72b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0627fc5bfb4f509fa1e95f14376e90'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:29.962965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cfba063c-f6ac-4c48-875e-1eb58042f72b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0627fc5bfb4f509fa1e95f14376e90'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:29.963644 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cfba063c-f6ac-4c48-875e-1eb58042f72b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0627fc5bfb4f509fa1e95f14376e90'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:30.051147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cfba063c-f6ac-4c48-875e-1eb58042f72b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0627fc5bfb4f509fa1e95f14376e90'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:30.064289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-cfba063c-f6ac-4c48-875e-1eb58042f72b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0627fc5bfb4f509fa1e95f14376e90'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:30.070251 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1418/2863] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:09:29 2022] DELETE /identity/v3/users/2c0627fc5bfb4f509fa1e95f14376e90 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:09:30.521566 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ade590-3fee-4e4e-918d-27e56b89fee7 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:30.523275 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b3ade590-3fee-4e4e-918d-27e56b89fee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:30.537025 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ade590-3fee-4e4e-918d-27e56b89fee7 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:30.598845 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7aa12-2443-467d-a79f-c7066b75bb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:30.599519 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7aa12-2443-467d-a79f-c7066b75bb60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:30.599815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7aa12-2443-467d-a79f-c7066b75bb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:30.600075 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7aa12-2443-467d-a79f-c7066b75bb60 admin admin] PATH_INFO: `/v3/projects/399b4b2b4e5f42f2a517ac330bd8e58b` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:30.601229 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7aa12-2443-467d-a79f-c7066b75bb60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=399b4b2b4e5f42f2a517ac330bd8e58b)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:30.603153 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7aa12-2443-467d-a79f-c7066b75bb60 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:30.708940 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-eda7aa12-2443-467d-a79f-c7066b75bb60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399b4b2b4e5f42f2a517ac330bd8e58b', 'request_id': 'req-eda7aa12-2443-467d-a79f-c7066b75bb60'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:30.714373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-eda7aa12-2443-467d-a79f-c7066b75bb60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399b4b2b4e5f42f2a517ac330bd8e58b', 'request_id': 'req-eda7aa12-2443-467d-a79f-c7066b75bb60'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:30.782646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-eda7aa12-2443-467d-a79f-c7066b75bb60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399b4b2b4e5f42f2a517ac330bd8e58b', 'request_id': 'req-eda7aa12-2443-467d-a79f-c7066b75bb60'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:30.789931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:30.794486 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-eda7aa12-2443-467d-a79f-c7066b75bb60 admin admin] The token cache is being invalidate because project 399b4b2b4e5f42f2a517ac330bd8e58b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:09:30.794775 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-eda7aa12-2443-467d-a79f-c7066b75bb60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:30.795452 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-eda7aa12-2443-467d-a79f-c7066b75bb60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:30.796937 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1446/2864] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:09:30 2022] DELETE /identity/v3/projects/399b4b2b4e5f42f2a517ac330bd8e58b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:09:30.801783 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2ff59-c8d9-40a0-ae71-ceabb1c713d6 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:30.802275 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2ff59-c8d9-40a0-ae71-ceabb1c713d6 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:30.802400 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2ff59-c8d9-40a0-ae71-ceabb1c713d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:30.822315 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-35a2ff59-c8d9-40a0-ae71-ceabb1c713d6 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:30.823546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-35a2ff59-c8d9-40a0-ae71-ceabb1c713d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:30.853581 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:30.892697 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1419/2865] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:09:30.900521 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2da7a-1745-40a0-9f21-077c0d71de1d None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:30.905223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2da7a-1745-40a0-9f21-077c0d71de1d None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:30.905223 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2da7a-1745-40a0-9f21-077c0d71de1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:30.921836 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-70b2da7a-1745-40a0-9f21-077c0d71de1d None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:30.923049 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-70b2da7a-1745-40a0-9f21-077c0d71de1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:30.931462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1447/2866] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:09:30.935540 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2ff59-c8d9-40a0-ae71-ceabb1c713d6 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:30.945820 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2ff59-c8d9-40a0-ae71-ceabb1c713d6 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:30.950341 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f5e4c-a947-48c1-88be-b9fbcb5d956a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:30.950938 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f5e4c-a947-48c1-88be-b9fbcb5d956a admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:30.951253 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f5e4c-a947-48c1-88be-b9fbcb5d956a admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:30.952153 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f5e4c-a947-48c1-88be-b9fbcb5d956a admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:30.952889 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f5e4c-a947-48c1-88be-b9fbcb5d956a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:09:30.953193 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f5e4c-a947-48c1-88be-b9fbcb5d956a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:30.954803 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f5e4c-a947-48c1-88be-b9fbcb5d956a admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:30.966301 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1420/2867] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:09:30.970526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2da7a-1745-40a0-9f21-077c0d71de1d None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:30.979733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2da7a-1745-40a0-9f21-077c0d71de1d None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:30.984487 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0af143-2ffa-440c-8af6-da74fd76975f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:30.985594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0af143-2ffa-440c-8af6-da74fd76975f admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:30.985594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0af143-2ffa-440c-8af6-da74fd76975f admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:30.985594 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0af143-2ffa-440c-8af6-da74fd76975f admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:30.986210 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0af143-2ffa-440c-8af6-da74fd76975f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:30.988128 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0af143-2ffa-440c-8af6-da74fd76975f admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:31.081974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1448/2868] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:09:30 2022] POST /identity/v3/projects => generated 401 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:09:31.085876 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f5e4c-a947-48c1-88be-b9fbcb5d956a admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:31.096218 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f5e4c-a947-48c1-88be-b9fbcb5d956a admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:31.151284 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7c6d8-9dd6-4314-a86d-d15b95aa115c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:31.151872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7c6d8-9dd6-4314-a86d-d15b95aa115c admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:31.152152 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7c6d8-9dd6-4314-a86d-d15b95aa115c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:31.152421 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7c6d8-9dd6-4314-a86d-d15b95aa115c admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:31.153042 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7c6d8-9dd6-4314-a86d-d15b95aa115c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:31.154973 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7c6d8-9dd6-4314-a86d-d15b95aa115c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:31.264338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1421/2869] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:09:31 2022] POST /identity/v3/users => generated 301 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:09:31.278012 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0af143-2ffa-440c-8af6-da74fd76975f admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:31.278919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0af143-2ffa-440c-8af6-da74fd76975f admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:31.283567 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c629f-9768-4b48-90c1-bdcd2d1715d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:31.284181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c629f-9768-4b48-90c1-bdcd2d1715d4 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:31.284462 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c629f-9768-4b48-90c1-bdcd2d1715d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:31.284733 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c629f-9768-4b48-90c1-bdcd2d1715d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:31.285373 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c629f-9768-4b48-90c1-bdcd2d1715d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:31.286970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c629f-9768-4b48-90c1-bdcd2d1715d4 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:31.297426 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1449/2870] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:09:31.301386 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7c6d8-9dd6-4314-a86d-d15b95aa115c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:31.311927 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7c6d8-9dd6-4314-a86d-d15b95aa115c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:31.316667 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fbcce-54a7-44b6-8112-085a7fb77430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:31.317281 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fbcce-54a7-44b6-8112-085a7fb77430 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:31.317557 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fbcce-54a7-44b6-8112-085a7fb77430 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:31.317824 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fbcce-54a7-44b6-8112-085a7fb77430 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:31.318458 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fbcce-54a7-44b6-8112-085a7fb77430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:31.320077 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fbcce-54a7-44b6-8112-085a7fb77430 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:31.330397 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1422/2871] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:09:31.334346 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c629f-9768-4b48-90c1-bdcd2d1715d4 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:31.344348 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c629f-9768-4b48-90c1-bdcd2d1715d4 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:31.348935 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d37247-12bf-4500-ae7e-afcb569e764e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:31.349562 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d37247-12bf-4500-ae7e-afcb569e764e admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:31.349840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d37247-12bf-4500-ae7e-afcb569e764e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:31.350113 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d37247-12bf-4500-ae7e-afcb569e764e admin admin] PATH_INFO: `/v3/projects/0270695680774c3ab0fbe01ccf657a65/users/71a7e5293c6b44cf8a5c24db5fd4ee94/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:31.364785 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d37247-12bf-4500-ae7e-afcb569e764e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0270695680774c3ab0fbe01ccf657a65, user_id=71a7e5293c6b44cf8a5c24db5fd4ee94, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:31.367458 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d37247-12bf-4500-ae7e-afcb569e764e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:31.440023 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1450/2872] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:09:31 2022] PUT /identity/v3/projects/0270695680774c3ab0fbe01ccf657a65/users/71a7e5293c6b44cf8a5c24db5fd4ee94/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:09:31.456646 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fed75-2b1c-48b1-98e5-78e081f172d7 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:31.456930 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fed75-2b1c-48b1-98e5-78e081f172d7 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:31.457203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fed75-2b1c-48b1-98e5-78e081f172d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:31.485738 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-6f6fed75-2b1c-48b1-98e5-78e081f172d7 None None] MFA Rules not processed for user `71a7e5293c6b44cf8a5c24db5fd4ee94`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:31.486931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6f6fed75-2b1c-48b1-98e5-78e081f172d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:31.555449 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1423/2873] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:09:31 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:09:31.560055 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d37247-12bf-4500-ae7e-afcb569e764e admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:31.625923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d37247-12bf-4500-ae7e-afcb569e764e admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:31.630509 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd8c44-8430-4f81-bdae-e9eb7b76fa87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:31.631117 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd8c44-8430-4f81-bdae-e9eb7b76fa87 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:31.631410 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd8c44-8430-4f81-bdae-e9eb7b76fa87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:31.631681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd8c44-8430-4f81-bdae-e9eb7b76fa87 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:31.632308 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd8c44-8430-4f81-bdae-e9eb7b76fa87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:31.634245 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd8c44-8430-4f81-bdae-e9eb7b76fa87 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:31.766183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1451/2874] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:09:31 2022] POST /identity/v3/projects => generated 401 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:09:31.770241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fed75-2b1c-48b1-98e5-78e081f172d7 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:31.781378 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fed75-2b1c-48b1-98e5-78e081f172d7 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:31.807202 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:31.835335 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123061b0-9827-45df-9565-6fd90e8e92ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:31.835931 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123061b0-9827-45df-9565-6fd90e8e92ee admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:31.836222 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123061b0-9827-45df-9565-6fd90e8e92ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:31.836490 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123061b0-9827-45df-9565-6fd90e8e92ee admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:31.837110 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123061b0-9827-45df-9565-6fd90e8e92ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:31.839038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123061b0-9827-45df-9565-6fd90e8e92ee admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:31.942230 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1424/2875] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:09:31 2022] POST /identity/v3/users => generated 307 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:09:31.946324 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd8c44-8430-4f81-bdae-e9eb7b76fa87 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:31.957187 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd8c44-8430-4f81-bdae-e9eb7b76fa87 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:31.961869 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1131d-03f2-440e-813d-cfbd88ecb391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:31.962453 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1131d-03f2-440e-813d-cfbd88ecb391 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:31.962734 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1131d-03f2-440e-813d-cfbd88ecb391 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:31.963003 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1131d-03f2-440e-813d-cfbd88ecb391 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:31.963677 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1131d-03f2-440e-813d-cfbd88ecb391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:31.965527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1131d-03f2-440e-813d-cfbd88ecb391 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:31.976647 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1452/2876] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:09:31.980678 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123061b0-9827-45df-9565-6fd90e8e92ee admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:31.990619 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123061b0-9827-45df-9565-6fd90e8e92ee admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:31.995291 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3a35c-2a5b-422e-8c8f-0a969070a132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:31.995974 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3a35c-2a5b-422e-8c8f-0a969070a132 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:31.996267 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3a35c-2a5b-422e-8c8f-0a969070a132 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:31.996543 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3a35c-2a5b-422e-8c8f-0a969070a132 admin admin] PATH_INFO: `/v3/projects/dee77f41e39247f2bc80cd4a7d72ed17/users/522582e2a39a449585d5a4fba3ab23a2/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:32.010875 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3a35c-2a5b-422e-8c8f-0a969070a132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dee77f41e39247f2bc80cd4a7d72ed17, user_id=522582e2a39a449585d5a4fba3ab23a2, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:32.013577 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3a35c-2a5b-422e-8c8f-0a969070a132 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:32.091823 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1425/2877] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:09:31 2022] PUT /identity/v3/projects/dee77f41e39247f2bc80cd4a7d72ed17/users/522582e2a39a449585d5a4fba3ab23a2/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:09:32.096317 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1131d-03f2-440e-813d-cfbd88ecb391 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:32.106171 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1131d-03f2-440e-813d-cfbd88ecb391 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:32.162183 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d1c84-3c00-411a-bf4b-a920a25b81e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:32.162779 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d1c84-3c00-411a-bf4b-a920a25b81e9 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:32.163055 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d1c84-3c00-411a-bf4b-a920a25b81e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:32.163352 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d1c84-3c00-411a-bf4b-a920a25b81e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:32.163981 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d1c84-3c00-411a-bf4b-a920a25b81e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:32.165559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d1c84-3c00-411a-bf4b-a920a25b81e9 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:32.175394 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1453/2878] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:09:32.179546 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3a35c-2a5b-422e-8c8f-0a969070a132 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:32.189902 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3a35c-2a5b-422e-8c8f-0a969070a132 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:32.194653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757cce1-6d36-4592-b439-a7068f009682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:32.195336 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757cce1-6d36-4592-b439-a7068f009682 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:32.195616 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757cce1-6d36-4592-b439-a7068f009682 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:32.195889 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757cce1-6d36-4592-b439-a7068f009682 admin admin] PATH_INFO: `/v3/projects/dee77f41e39247f2bc80cd4a7d72ed17/users/522582e2a39a449585d5a4fba3ab23a2/roles/d6822e7f42594dc4ba880ca6b9e13e0c` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:32.197789 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757cce1-6d36-4592-b439-a7068f009682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dee77f41e39247f2bc80cd4a7d72ed17, user_id=522582e2a39a449585d5a4fba3ab23a2, role_id=d6822e7f42594dc4ba880ca6b9e13e0c)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:32.200482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757cce1-6d36-4592-b439-a7068f009682 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:32.211338 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1426/2879] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:09:32 2022] PUT /identity/v3/projects/dee77f41e39247f2bc80cd4a7d72ed17/users/522582e2a39a449585d5a4fba3ab23a2/roles/d6822e7f42594dc4ba880ca6b9e13e0c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:09:32.222412 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7a941-4719-4400-a9a1-74bdcd10cce0 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:32.222697 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7a941-4719-4400-a9a1-74bdcd10cce0 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:32.222975 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7a941-4719-4400-a9a1-74bdcd10cce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:32.231084 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:32.251404 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-35f7a941-4719-4400-a9a1-74bdcd10cce0 None None] MFA Rules not processed for user `522582e2a39a449585d5a4fba3ab23a2`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:32.252648 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-35f7a941-4719-4400-a9a1-74bdcd10cce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:32.319108 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1454/2880] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:09:32 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:09:32.330278 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757cce1-6d36-4592-b439-a7068f009682 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:32.331962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-c757cce1-6d36-4592-b439-a7068f009682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:32.388161 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757cce1-6d36-4592-b439-a7068f009682 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:32.392566 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146d983-7213-4f5d-b6d8-a20bff219288 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:32.397167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146d983-7213-4f5d-b6d8-a20bff219288 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:32.397167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146d983-7213-4f5d-b6d8-a20bff219288 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:32.397167 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146d983-7213-4f5d-b6d8-a20bff219288 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:32.402810 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146d983-7213-4f5d-b6d8-a20bff219288 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:32.404320 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146d983-7213-4f5d-b6d8-a20bff219288 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:32.413635 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1427/2881] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:09:32 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:09:32.625939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7a941-4719-4400-a9a1-74bdcd10cce0 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:32.635839 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7a941-4719-4400-a9a1-74bdcd10cce0 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:32.639366 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc59fe7-45f7-4ee6-b4dc-9cc41a859334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:32.639960 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc59fe7-45f7-4ee6-b4dc-9cc41a859334 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:32.646857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc59fe7-45f7-4ee6-b4dc-9cc41a859334 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:32.646857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc59fe7-45f7-4ee6-b4dc-9cc41a859334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:32.697051 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc59fe7-45f7-4ee6-b4dc-9cc41a859334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:32.698479 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc59fe7-45f7-4ee6-b4dc-9cc41a859334 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:32.707186 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1455/2882] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:09:32 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:09:37.700213 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146d983-7213-4f5d-b6d8-a20bff219288 service cinder] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:37.760924 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146d983-7213-4f5d-b6d8-a20bff219288 service cinder] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:37.766086 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4a684-b864-43ff-bfac-9d633534bb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:37.766705 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4a684-b864-43ff-bfac-9d633534bb4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:37.766995 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4a684-b864-43ff-bfac-9d633534bb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:37.767280 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4a684-b864-43ff-bfac-9d633534bb4f admin admin] PATH_INFO: `/v3/users/71a7e5293c6b44cf8a5c24db5fd4ee94` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:37.768395 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c4a684-b864-43ff-bfac-9d633534bb4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=71a7e5293c6b44cf8a5c24db5fd4ee94)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:37.770302 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c4a684-b864-43ff-bfac-9d633534bb4f admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:37.863675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:37.961721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-36c4a684-b864-43ff-bfac-9d633534bb4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71a7e5293c6b44cf8a5c24db5fd4ee94'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:37.968723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-36c4a684-b864-43ff-bfac-9d633534bb4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71a7e5293c6b44cf8a5c24db5fd4ee94'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:37.969389 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-36c4a684-b864-43ff-bfac-9d633534bb4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71a7e5293c6b44cf8a5c24db5fd4ee94'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:38.045347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-36c4a684-b864-43ff-bfac-9d633534bb4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71a7e5293c6b44cf8a5c24db5fd4ee94'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:38.057702 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-36c4a684-b864-43ff-bfac-9d633534bb4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71a7e5293c6b44cf8a5c24db5fd4ee94'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:38.064155 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1428/2883] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:09:37 2022] DELETE /identity/v3/users/71a7e5293c6b44cf8a5c24db5fd4ee94 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:09:38.068633 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31367d3c-8894-4cd5-8378-b3892e4d94a5 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:38.069114 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31367d3c-8894-4cd5-8378-b3892e4d94a5 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:38.069225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31367d3c-8894-4cd5-8378-b3892e4d94a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:38.087918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-31367d3c-8894-4cd5-8378-b3892e4d94a5 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:38.090087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-31367d3c-8894-4cd5-8378-b3892e4d94a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:38.146323 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1456/2884] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:09:38.155847 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4a684-b864-43ff-bfac-9d633534bb4f admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:38.157237 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-36c4a684-b864-43ff-bfac-9d633534bb4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:38.213520 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4a684-b864-43ff-bfac-9d633534bb4f admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:38.217467 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19d04e-9c55-4880-b7ea-f64116bd24c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:38.218561 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19d04e-9c55-4880-b7ea-f64116bd24c6 service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:38.218838 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19d04e-9c55-4880-b7ea-f64116bd24c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:38.219128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19d04e-9c55-4880-b7ea-f64116bd24c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:38.227996 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19d04e-9c55-4880-b7ea-f64116bd24c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:38.229431 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19d04e-9c55-4880-b7ea-f64116bd24c6 service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:38.239377 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1429/2885] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:09:38.651878 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31367d3c-8894-4cd5-8378-b3892e4d94a5 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:38.653276 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-31367d3c-8894-4cd5-8378-b3892e4d94a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:38.666040 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31367d3c-8894-4cd5-8378-b3892e4d94a5 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:38.670781 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd7acb-b256-4b63-825d-7fe55d9e4804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:38.671736 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd7acb-b256-4b63-825d-7fe55d9e4804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:38.672009 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd7acb-b256-4b63-825d-7fe55d9e4804 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:38.672284 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd7acb-b256-4b63-825d-7fe55d9e4804 admin admin] PATH_INFO: `/v3/projects/0270695680774c3ab0fbe01ccf657a65` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:38.673393 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd7acb-b256-4b63-825d-7fe55d9e4804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0270695680774c3ab0fbe01ccf657a65)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:38.675318 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd7acb-b256-4b63-825d-7fe55d9e4804 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:38.701358 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:38.810913 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-8dfd7acb-b256-4b63-825d-7fe55d9e4804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0270695680774c3ab0fbe01ccf657a65', 'request_id': 'req-8dfd7acb-b256-4b63-825d-7fe55d9e4804'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:38.817067 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-8dfd7acb-b256-4b63-825d-7fe55d9e4804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0270695680774c3ab0fbe01ccf657a65', 'request_id': 'req-8dfd7acb-b256-4b63-825d-7fe55d9e4804'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:38.902866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-8dfd7acb-b256-4b63-825d-7fe55d9e4804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0270695680774c3ab0fbe01ccf657a65', 'request_id': 'req-8dfd7acb-b256-4b63-825d-7fe55d9e4804'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:38.915030 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-8dfd7acb-b256-4b63-825d-7fe55d9e4804 admin admin] The token cache is being invalidate because project 0270695680774c3ab0fbe01ccf657a65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:09:38.919341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-8dfd7acb-b256-4b63-825d-7fe55d9e4804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:38.919341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-8dfd7acb-b256-4b63-825d-7fe55d9e4804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:38.919341 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1457/2886] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:09:38 2022] DELETE /identity/v3/projects/0270695680774c3ab0fbe01ccf657a65 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:09:38.921482 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19d04e-9c55-4880-b7ea-f64116bd24c6 service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:38.931276 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19d04e-9c55-4880-b7ea-f64116bd24c6 service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:38.987685 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26949e4-c49f-4744-bed6-014adb2a6a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:38.988384 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26949e4-c49f-4744-bed6-014adb2a6a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:38.988664 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26949e4-c49f-4744-bed6-014adb2a6a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:38.988942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26949e4-c49f-4744-bed6-014adb2a6a31 admin admin] PATH_INFO: `/v3/users/522582e2a39a449585d5a4fba3ab23a2` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:38.990095 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26949e4-c49f-4744-bed6-014adb2a6a31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=522582e2a39a449585d5a4fba3ab23a2)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:38.992089 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26949e4-c49f-4744-bed6-014adb2a6a31 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:39.150088 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d26949e4-c49f-4744-bed6-014adb2a6a31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '522582e2a39a449585d5a4fba3ab23a2'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:39.151274 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:39.157289 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d26949e4-c49f-4744-bed6-014adb2a6a31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '522582e2a39a449585d5a4fba3ab23a2'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:39.157946 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d26949e4-c49f-4744-bed6-014adb2a6a31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '522582e2a39a449585d5a4fba3ab23a2'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:39.216541 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d26949e4-c49f-4744-bed6-014adb2a6a31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '522582e2a39a449585d5a4fba3ab23a2'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:39.228508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-d26949e4-c49f-4744-bed6-014adb2a6a31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '522582e2a39a449585d5a4fba3ab23a2'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:39.236880 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1430/2887] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:09:38 2022] DELETE /identity/v3/users/522582e2a39a449585d5a4fba3ab23a2 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:09:39.663362 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd7acb-b256-4b63-825d-7fe55d9e4804 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:39.664825 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-8dfd7acb-b256-4b63-825d-7fe55d9e4804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:39.678557 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd7acb-b256-4b63-825d-7fe55d9e4804 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:39.736556 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed023740-6fb3-473c-b23d-ebb128ca40e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:39.737181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed023740-6fb3-473c-b23d-ebb128ca40e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:39.737457 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed023740-6fb3-473c-b23d-ebb128ca40e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:39.737726 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed023740-6fb3-473c-b23d-ebb128ca40e2 admin admin] PATH_INFO: `/v3/projects/dee77f41e39247f2bc80cd4a7d72ed17` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:39.738817 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed023740-6fb3-473c-b23d-ebb128ca40e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dee77f41e39247f2bc80cd4a7d72ed17)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:39.740803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed023740-6fb3-473c-b23d-ebb128ca40e2 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:39.885359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ed023740-6fb3-473c-b23d-ebb128ca40e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dee77f41e39247f2bc80cd4a7d72ed17', 'request_id': 'req-ed023740-6fb3-473c-b23d-ebb128ca40e2'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:39.890938 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ed023740-6fb3-473c-b23d-ebb128ca40e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dee77f41e39247f2bc80cd4a7d72ed17', 'request_id': 'req-ed023740-6fb3-473c-b23d-ebb128ca40e2'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:39.971157 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ed023740-6fb3-473c-b23d-ebb128ca40e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dee77f41e39247f2bc80cd4a7d72ed17', 'request_id': 'req-ed023740-6fb3-473c-b23d-ebb128ca40e2'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:39.982422 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ed023740-6fb3-473c-b23d-ebb128ca40e2 admin admin] The token cache is being invalidate because project dee77f41e39247f2bc80cd4a7d72ed17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:09:39.982705 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ed023740-6fb3-473c-b23d-ebb128ca40e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:39.983967 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-ed023740-6fb3-473c-b23d-ebb128ca40e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:09:39.985437 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1458/2888] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:09:39 2022] DELETE /identity/v3/projects/dee77f41e39247f2bc80cd4a7d72ed17 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:09:39.990383 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f3e69-6776-4364-92f3-268fcc166124 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:39.990894 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f3e69-6776-4364-92f3-268fcc166124 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:39.991001 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f3e69-6776-4364-92f3-268fcc166124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:40.010589 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-239f3e69-6776-4364-92f3-268fcc166124 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:40.011852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-239f3e69-6776-4364-92f3-268fcc166124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:40.069241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1431/2889] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:09:40.076896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5729c77-8529-414f-b94c-526ceb36a652 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:40.077706 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5729c77-8529-414f-b94c-526ceb36a652 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:40.077974 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5729c77-8529-414f-b94c-526ceb36a652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:40.098091 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-b5729c77-8529-414f-b94c-526ceb36a652 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:40.099359 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-b5729c77-8529-414f-b94c-526ceb36a652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:40.107401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1459/2890] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:09:40.111516 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f3e69-6776-4364-92f3-268fcc166124 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:40.121065 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f3e69-6776-4364-92f3-268fcc166124 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:40.125554 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c420ae-1f26-4453-a00f-3432be19e59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:40.126135 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c420ae-1f26-4453-a00f-3432be19e59c admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:40.126411 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c420ae-1f26-4453-a00f-3432be19e59c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:40.126682 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c420ae-1f26-4453-a00f-3432be19e59c admin admin] PATH_INFO: `/v3/domains` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:40.127366 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c420ae-1f26-4453-a00f-3432be19e59c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88059) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:09:40.127652 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c420ae-1f26-4453-a00f-3432be19e59c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:40.129241 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c420ae-1f26-4453-a00f-3432be19e59c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:40.140869 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1432/2891] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:09:40.144887 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5729c77-8529-414f-b94c-526ceb36a652 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:40.154369 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5729c77-8529-414f-b94c-526ceb36a652 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:40.159671 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb8823-1df7-4c14-87ec-be0b35ef9093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:40.164877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb8823-1df7-4c14-87ec-be0b35ef9093 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:40.164877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb8823-1df7-4c14-87ec-be0b35ef9093 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:40.164877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb8823-1df7-4c14-87ec-be0b35ef9093 admin admin] PATH_INFO: `/v3/projects` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:40.164877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb8823-1df7-4c14-87ec-be0b35ef9093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:40.164877 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb8823-1df7-4c14-87ec-be0b35ef9093 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:40.252914 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1460/2892] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:09:40 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:09:40.257147 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c420ae-1f26-4453-a00f-3432be19e59c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:40.267786 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c420ae-1f26-4453-a00f-3432be19e59c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:40.322965 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed5dad-efa3-4c65-8c1f-e946eb27c934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:40.323591 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed5dad-efa3-4c65-8c1f-e946eb27c934 admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:40.323866 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed5dad-efa3-4c65-8c1f-e946eb27c934 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:40.324132 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed5dad-efa3-4c65-8c1f-e946eb27c934 admin admin] PATH_INFO: `/v3/users` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:40.324762 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed5dad-efa3-4c65-8c1f-e946eb27c934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:40.326689 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed5dad-efa3-4c65-8c1f-e946eb27c934 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:40.393979 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1433/2893] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:09:40 2022] POST /identity/v3/users => generated 286 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:09:40.398292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb8823-1df7-4c14-87ec-be0b35ef9093 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:40.408646 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb8823-1df7-4c14-87ec-be0b35ef9093 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:40.413593 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c26488f-69bd-4b08-862c-5c119b28d30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:40.414190 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26488f-69bd-4b08-862c-5c119b28d30e admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:40.414469 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26488f-69bd-4b08-862c-5c119b28d30e admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:40.414739 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26488f-69bd-4b08-862c-5c119b28d30e admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:40.415446 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c26488f-69bd-4b08-862c-5c119b28d30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:40.417045 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c26488f-69bd-4b08-862c-5c119b28d30e admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:40.427380 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1461/2894] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:09:40.431379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed5dad-efa3-4c65-8c1f-e946eb27c934 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:40.441721 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed5dad-efa3-4c65-8c1f-e946eb27c934 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:40.446407 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120117d9-1630-47e8-9511-f0462616a8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:40.446987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120117d9-1630-47e8-9511-f0462616a8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:40.447295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120117d9-1630-47e8-9511-f0462616a8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:40.447569 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120117d9-1630-47e8-9511-f0462616a8d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:40.448203 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120117d9-1630-47e8-9511-f0462616a8d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:40.449800 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120117d9-1630-47e8-9511-f0462616a8d9 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:40.460218 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1434/2895] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:09:40.464284 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c26488f-69bd-4b08-862c-5c119b28d30e admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:40.470217 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:40.475477 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c26488f-69bd-4b08-862c-5c119b28d30e admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:40.480285 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d4cee-6d70-4246-b605-0dcdea779baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:40.480927 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d4cee-6d70-4246-b605-0dcdea779baf admin admin] REQUEST_METHOD: `PUT` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:40.481207 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d4cee-6d70-4246-b605-0dcdea779baf admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:40.481484 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d4cee-6d70-4246-b605-0dcdea779baf admin admin] PATH_INFO: `/v3/projects/266be8998b214850a9bd63c50e8fecd9/users/cdc5bfcb702f4768bcc6ef35c8f443b0/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:40.496599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d4cee-6d70-4246-b605-0dcdea779baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=266be8998b214850a9bd63c50e8fecd9, user_id=cdc5bfcb702f4768bcc6ef35c8f443b0, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:40.499279 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d4cee-6d70-4246-b605-0dcdea779baf admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:40.570998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1462/2896] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:09:40 2022] PUT /identity/v3/projects/266be8998b214850a9bd63c50e8fecd9/users/cdc5bfcb702f4768bcc6ef35c8f443b0/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:09:40.576093 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cdf60-b993-4db4-86e0-69ef434474d4 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:40.576379 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cdf60-b993-4db4-86e0-69ef434474d4 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:40.576657 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cdf60-b993-4db4-86e0-69ef434474d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:40.596400 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-2c1cdf60-b993-4db4-86e0-69ef434474d4 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:40.597602 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-2c1cdf60-b993-4db4-86e0-69ef434474d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:40.623420 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:40.655884 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1435/2897] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:09:40.665654 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d4cee-6d70-4246-b605-0dcdea779baf admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:40.667555 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-ee2d4cee-6d70-4246-b605-0dcdea779baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:40.724996 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d4cee-6d70-4246-b605-0dcdea779baf admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:40.728667 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3588f7-4967-4f4f-a078-22502d3456a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:40.729255 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3588f7-4967-4f4f-a078-22502d3456a1 service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:40.729530 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3588f7-4967-4f4f-a078-22502d3456a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:40.729798 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3588f7-4967-4f4f-a078-22502d3456a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:40.738650 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3588f7-4967-4f4f-a078-22502d3456a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:40.740096 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3588f7-4967-4f4f-a078-22502d3456a1 service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:40.749777 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1463/2898] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:09:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:09:42.055128 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660d8a6-d318-46ba-95ba-27efff3cb55d None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:42.055617 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660d8a6-d318-46ba-95ba-27efff3cb55d None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:42.055710 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660d8a6-d318-46ba-95ba-27efff3cb55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:42.088231 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-a660d8a6-d318-46ba-95ba-27efff3cb55d None None] MFA Rules not processed for user `cdc5bfcb702f4768bcc6ef35c8f443b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:42.089460 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a660d8a6-d318-46ba-95ba-27efff3cb55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:42.167260 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1436/2899] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:09:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:09:42.175526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3588f7-4967-4f4f-a078-22502d3456a1 service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:42.176918 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-aa3588f7-4967-4f4f-a078-22502d3456a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:42.235536 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3588f7-4967-4f4f-a078-22502d3456a1 service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:42.239221 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e1771-2771-4bb6-9f9e-7400b95966a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:42.239852 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e1771-2771-4bb6-9f9e-7400b95966a2 service cinder] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:42.240130 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e1771-2771-4bb6-9f9e-7400b95966a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:42.240401 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e1771-2771-4bb6-9f9e-7400b95966a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:42.249093 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e1771-2771-4bb6-9f9e-7400b95966a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:42.250511 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e1771-2771-4bb6-9f9e-7400b95966a2 service cinder] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:42.259272 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1464/2900] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:09:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:09:43.970084 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29ec74-84f0-40d0-89c0-1866b8a296b6 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:43.970600 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29ec74-84f0-40d0-89c0-1866b8a296b6 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:43.970712 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29ec74-84f0-40d0-89c0-1866b8a296b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:43.990653 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-ad29ec74-84f0-40d0-89c0-1866b8a296b6 None None] MFA Rules not processed for user `cdc5bfcb702f4768bcc6ef35c8f443b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:43.992158 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ad29ec74-84f0-40d0-89c0-1866b8a296b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:43.998699 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1437/2901] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:09:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:09:45.762129 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e1771-2771-4bb6-9f9e-7400b95966a2 service cinder] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:45.763681 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-3a1e1771-2771-4bb6-9f9e-7400b95966a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:45.832828 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e1771-2771-4bb6-9f9e-7400b95966a2 service cinder] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:45.837896 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468959a-f9f4-4794-809e-2fcd593605ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4f11655cd51438e9652869f74ffa975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:45.838497 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468959a-f9f4-4794-809e-2fcd593605ee service placement] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:45.838766 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468959a-f9f4-4794-809e-2fcd593605ee service placement] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:45.839032 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468959a-f9f4-4794-809e-2fcd593605ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:45.841019 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-7468959a-f9f4-4794-809e-2fcd593605ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:45.853281 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468959a-f9f4-4794-809e-2fcd593605ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:45.854880 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468959a-f9f4-4794-809e-2fcd593605ee service placement] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:45.864824 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1465/2902] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:09:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:09:53.984062 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=48 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:55.860448 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=93,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:10:00.177627 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29ec74-84f0-40d0-89c0-1866b8a296b6 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:00.179154 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-ad29ec74-84f0-40d0-89c0-1866b8a296b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:00.269330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29ec74-84f0-40d0-89c0-1866b8a296b6 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:00.275149 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd095131-6a33-40af-84ac-f34d9ec4bca9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b6e4c5bf2a54b28a55e5ae74b90ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:00.275770 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd095131-6a33-40af-84ac-f34d9ec4bca9 service nova] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:00.276049 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd095131-6a33-40af-84ac-f34d9ec4bca9 service nova] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:00.276323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd095131-6a33-40af-84ac-f34d9ec4bca9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:00.278084 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-dd095131-6a33-40af-84ac-f34d9ec4bca9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:00.294024 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd095131-6a33-40af-84ac-f34d9ec4bca9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:00.296287 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd095131-6a33-40af-84ac-f34d9ec4bca9 service nova] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:00.307479 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1438/2903] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:10:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:10:00.801512 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca5055-a26a-4c40-ae5a-42cb9ac28361 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:00.802120 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca5055-a26a-4c40-ae5a-42cb9ac28361 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:00.802120 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca5055-a26a-4c40-ae5a-42cb9ac28361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:00.823524 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-fdca5055-a26a-4c40-ae5a-42cb9ac28361 None None] MFA Rules not processed for user `cdc5bfcb702f4768bcc6ef35c8f443b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:10:00.825052 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-fdca5055-a26a-4c40-ae5a-42cb9ac28361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:00.834039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1466/2904] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:10:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:10:01.090347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd095131-6a33-40af-84ac-f34d9ec4bca9 service nova] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:01.091804 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-dd095131-6a33-40af-84ac-f34d9ec4bca9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:01.103566 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd095131-6a33-40af-84ac-f34d9ec4bca9 service nova] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:01.107347 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70083221-098a-410c-a412-ac49b3863e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:01.107988 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70083221-098a-410c-a412-ac49b3863e6e service neutron] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:01.108796 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70083221-098a-410c-a412-ac49b3863e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:01.109225 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70083221-098a-410c-a412-ac49b3863e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:01.111011 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-70083221-098a-410c-a412-ac49b3863e6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:01.121396 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70083221-098a-410c-a412-ac49b3863e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:01.122835 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70083221-098a-410c-a412-ac49b3863e6e service neutron] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:01.132410 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1439/2905] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:10:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:10:09.199192 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca5055-a26a-4c40-ae5a-42cb9ac28361 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:09.200723 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-fdca5055-a26a-4c40-ae5a-42cb9ac28361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:09.216844 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca5055-a26a-4c40-ae5a-42cb9ac28361 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:09.222072 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97508763-767e-4ccc-8e5a-16f686409b59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4f11655cd51438e9652869f74ffa975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:09.222673 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97508763-767e-4ccc-8e5a-16f686409b59 service placement] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:09.223022 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97508763-767e-4ccc-8e5a-16f686409b59 service placement] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:09.223328 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97508763-767e-4ccc-8e5a-16f686409b59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:09.225163 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-97508763-767e-4ccc-8e5a-16f686409b59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:09.238945 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97508763-767e-4ccc-8e5a-16f686409b59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:09.242244 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97508763-767e-4ccc-8e5a-16f686409b59 service placement] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:09.251467 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1467/2906] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:10:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:10:11.128329 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=40 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:10:13.388283 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70083221-098a-410c-a412-ac49b3863e6e service neutron] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:13.390432 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-70083221-098a-410c-a412-ac49b3863e6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:13.407598 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70083221-098a-410c-a412-ac49b3863e6e service neutron] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:13.412329 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b4369-0fd4-4a93-9c1c-21a6be7a5211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:13.412962 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b4369-0fd4-4a93-9c1c-21a6be7a5211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:13.413238 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b4369-0fd4-4a93-9c1c-21a6be7a5211 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:13.413507 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b4369-0fd4-4a93-9c1c-21a6be7a5211 admin admin] PATH_INFO: `/v3/users/dbaf48d78bc143d1b3884816a6f3c3f7` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:13.414673 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b4369-0fd4-4a93-9c1c-21a6be7a5211 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbaf48d78bc143d1b3884816a6f3c3f7)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:13.416632 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b4369-0fd4-4a93-9c1c-21a6be7a5211 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:13.594991 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-062b4369-0fd4-4a93-9c1c-21a6be7a5211 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbaf48d78bc143d1b3884816a6f3c3f7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:13.602300 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-062b4369-0fd4-4a93-9c1c-21a6be7a5211 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbaf48d78bc143d1b3884816a6f3c3f7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:13.602942 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-062b4369-0fd4-4a93-9c1c-21a6be7a5211 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbaf48d78bc143d1b3884816a6f3c3f7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:13.707313 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-062b4369-0fd4-4a93-9c1c-21a6be7a5211 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbaf48d78bc143d1b3884816a6f3c3f7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:13.720877 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-062b4369-0fd4-4a93-9c1c-21a6be7a5211 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbaf48d78bc143d1b3884816a6f3c3f7'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:13.727126 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1440/2907] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:10:13 2022] DELETE /identity/v3/users/dbaf48d78bc143d1b3884816a6f3c3f7 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:10:14.011424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97508763-767e-4ccc-8e5a-16f686409b59 service placement] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:14.022563 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97508763-767e-4ccc-8e5a-16f686409b59 service placement] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:14.084133 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f83df6-b40a-4852-9a12-7af38e0bc75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:14.084687 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f83df6-b40a-4852-9a12-7af38e0bc75d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:14.084960 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f83df6-b40a-4852-9a12-7af38e0bc75d admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:14.085230 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f83df6-b40a-4852-9a12-7af38e0bc75d admin admin] PATH_INFO: `/v3/projects/1fac10aecb7b4050ac8c85a40db19d55` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:14.086480 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f83df6-b40a-4852-9a12-7af38e0bc75d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fac10aecb7b4050ac8c85a40db19d55)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:14.088498 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f83df6-b40a-4852-9a12-7af38e0bc75d admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:14.198374 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-42f83df6-b40a-4852-9a12-7af38e0bc75d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fac10aecb7b4050ac8c85a40db19d55', 'request_id': 'req-42f83df6-b40a-4852-9a12-7af38e0bc75d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:14.203857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-42f83df6-b40a-4852-9a12-7af38e0bc75d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fac10aecb7b4050ac8c85a40db19d55', 'request_id': 'req-42f83df6-b40a-4852-9a12-7af38e0bc75d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:14.313084 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-42f83df6-b40a-4852-9a12-7af38e0bc75d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fac10aecb7b4050ac8c85a40db19d55', 'request_id': 'req-42f83df6-b40a-4852-9a12-7af38e0bc75d'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:14.324962 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-42f83df6-b40a-4852-9a12-7af38e0bc75d admin admin] The token cache is being invalidate because project 1fac10aecb7b4050ac8c85a40db19d55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:10:14.325251 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-42f83df6-b40a-4852-9a12-7af38e0bc75d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:14.325875 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-42f83df6-b40a-4852-9a12-7af38e0bc75d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:14.327377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1468/2908] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:10:14 2022] DELETE /identity/v3/projects/1fac10aecb7b4050ac8c85a40db19d55 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:10:14.331330 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b4369-0fd4-4a93-9c1c-21a6be7a5211 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:14.332723 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-062b4369-0fd4-4a93-9c1c-21a6be7a5211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:14.346747 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b4369-0fd4-4a93-9c1c-21a6be7a5211 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:14.402381 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767215d-dfae-4831-9982-3992e9895615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:14.403027 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767215d-dfae-4831-9982-3992e9895615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:14.405636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767215d-dfae-4831-9982-3992e9895615 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:14.405636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767215d-dfae-4831-9982-3992e9895615 admin admin] PATH_INFO: `/v3/users/aae21d12792e471e8e20adecd8482b14` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:14.405636 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8767215d-dfae-4831-9982-3992e9895615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aae21d12792e471e8e20adecd8482b14)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:14.406676 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8767215d-dfae-4831-9982-3992e9895615 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:14.577382 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8767215d-dfae-4831-9982-3992e9895615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aae21d12792e471e8e20adecd8482b14'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:14.584116 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8767215d-dfae-4831-9982-3992e9895615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aae21d12792e471e8e20adecd8482b14'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:14.584772 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8767215d-dfae-4831-9982-3992e9895615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aae21d12792e471e8e20adecd8482b14'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:14.655772 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8767215d-dfae-4831-9982-3992e9895615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aae21d12792e471e8e20adecd8482b14'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:14.664435 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:10:14.669234 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-8767215d-dfae-4831-9982-3992e9895615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aae21d12792e471e8e20adecd8482b14'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:14.675818 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1441/2909] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:10:14 2022] DELETE /identity/v3/users/aae21d12792e471e8e20adecd8482b14 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:10:14.942314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f83df6-b40a-4852-9a12-7af38e0bc75d admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:14.942314 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-42f83df6-b40a-4852-9a12-7af38e0bc75d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:14.954652 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f83df6-b40a-4852-9a12-7af38e0bc75d admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:15.044138 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:15.045580 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:15.045930 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:15.046815 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2 admin admin] PATH_INFO: `/v3/projects/1965018347464ce8b4c0f9e8923b81b1` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:15.048476 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1965018347464ce8b4c0f9e8923b81b1)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:15.050450 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:15.067037 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:10:15.169847 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1965018347464ce8b4c0f9e8923b81b1', 'request_id': 'req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:15.175970 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1965018347464ce8b4c0f9e8923b81b1', 'request_id': 'req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:15.255694 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1965018347464ce8b4c0f9e8923b81b1', 'request_id': 'req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:15.274239 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2 admin admin] The token cache is being invalidate because project 1965018347464ce8b4c0f9e8923b81b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:10:15.274526 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:15.275229 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:15.277181 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1469/2910] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:10:14 2022] DELETE /identity/v3/projects/1965018347464ce8b4c0f9e8923b81b1 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:10:21.952624 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bee8c-215a-40a4-a817-31db2d25ef17 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:21.953184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bee8c-215a-40a4-a817-31db2d25ef17 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:21.953184 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bee8c-215a-40a4-a817-31db2d25ef17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:22.001837 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-044bee8c-215a-40a4-a817-31db2d25ef17 None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:10:22.003016 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-044bee8c-215a-40a4-a817-31db2d25ef17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:22.067466 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1442/2911] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Aug 30 15:10:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:10:22.214141 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:22.215602 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:22.304344 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb2d0c-55e0-44f9-9507-2abd85633ee2 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:22.309312 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1beb28-a993-440b-86b2-bca9a3ea9ae7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:22.309904 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1beb28-a993-440b-86b2-bca9a3ea9ae7 service swift] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:22.310178 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1beb28-a993-440b-86b2-bca9a3ea9ae7 service swift] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:22.310445 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1beb28-a993-440b-86b2-bca9a3ea9ae7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:22.318455 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1beb28-a993-440b-86b2-bca9a3ea9ae7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:22.320039 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1beb28-a993-440b-86b2-bca9a3ea9ae7 service swift] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:22.327998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1470/2912] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 15:10:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:10:32.064510 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=43,keystone:DELETE=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:10:32.324659 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:10:38.958928 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d435849-f2d9-4c69-9914-5129f2a94504 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:38.959408 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d435849-f2d9-4c69-9914-5129f2a94504 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:38.959516 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d435849-f2d9-4c69-9914-5129f2a94504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:38.978688 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-4d435849-f2d9-4c69-9914-5129f2a94504 None None] MFA Rules not processed for user `912e9c732f194fa496a9111bbfab3efa`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:10:38.979918 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4d435849-f2d9-4c69-9914-5129f2a94504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:38.986973 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1443/2913] 10.222.0.27 () {58 vars in 967 bytes} [Tue Aug 30 15:10:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:10:38.998939 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1beb28-a993-440b-86b2-bca9a3ea9ae7 service swift] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:39.009965 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1beb28-a993-440b-86b2-bca9a3ea9ae7 service swift] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:39.014919 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef5451-d129-4cb7-ac11-148e1f823d72 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc23480d5aa94cad9e6599762ce01548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:39.015542 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef5451-d129-4cb7-ac11-148e1f823d72 service swift] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:39.015818 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef5451-d129-4cb7-ac11-148e1f823d72 service swift] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:39.016085 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef5451-d129-4cb7-ac11-148e1f823d72 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:39.024611 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef5451-d129-4cb7-ac11-148e1f823d72 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:39.026170 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef5451-d129-4cb7-ac11-148e1f823d72 service swift] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:39.034127 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1471/2914] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Aug 30 15:10:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:10:41.880023 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d435849-f2d9-4c69-9914-5129f2a94504 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:41.881451 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-4d435849-f2d9-4c69-9914-5129f2a94504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:41.949852 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d435849-f2d9-4c69-9914-5129f2a94504 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:41.954935 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741a706-268c-42ef-bc21-ee6b96a634ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4f11655cd51438e9652869f74ffa975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:41.955578 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741a706-268c-42ef-bc21-ee6b96a634ae service placement] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:41.955857 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741a706-268c-42ef-bc21-ee6b96a634ae service placement] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:41.956130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741a706-268c-42ef-bc21-ee6b96a634ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:41.958368 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-a741a706-268c-42ef-bc21-ee6b96a634ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:41.969971 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741a706-268c-42ef-bc21-ee6b96a634ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:41.971584 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741a706-268c-42ef-bc21-ee6b96a634ae service placement] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:41.981257 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1444/2915] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:10:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:10:42.022737 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef5451-d129-4cb7-ac11-148e1f823d72 service swift] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:42.024422 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-c0ef5451-d129-4cb7-ac11-148e1f823d72 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:42.089396 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef5451-d129-4cb7-ac11-148e1f823d72 service swift] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:42.093840 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93244a5-aac3-440f-848c-7221a7998da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:42.094455 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93244a5-aac3-440f-848c-7221a7998da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:42.094729 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93244a5-aac3-440f-848c-7221a7998da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:42.094996 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93244a5-aac3-440f-848c-7221a7998da4 admin admin] PATH_INFO: `/v3/users/cdc5bfcb702f4768bcc6ef35c8f443b0` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:42.096126 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93244a5-aac3-440f-848c-7221a7998da4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdc5bfcb702f4768bcc6ef35c8f443b0)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:42.098091 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93244a5-aac3-440f-848c-7221a7998da4 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:42.290857 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a93244a5-aac3-440f-848c-7221a7998da4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdc5bfcb702f4768bcc6ef35c8f443b0'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:42.297754 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a93244a5-aac3-440f-848c-7221a7998da4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdc5bfcb702f4768bcc6ef35c8f443b0'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:42.309931 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a93244a5-aac3-440f-848c-7221a7998da4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdc5bfcb702f4768bcc6ef35c8f443b0'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:42.310610 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a93244a5-aac3-440f-848c-7221a7998da4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdc5bfcb702f4768bcc6ef35c8f443b0'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:42.311273 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-a93244a5-aac3-440f-848c-7221a7998da4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdc5bfcb702f4768bcc6ef35c8f443b0'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:42.385826 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1472/2916] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:10:42 2022] DELETE /identity/v3/users/cdc5bfcb702f4768bcc6ef35c8f443b0 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:10:42.668044 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741a706-268c-42ef-bc21-ee6b96a634ae service placement] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:42.682654 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741a706-268c-42ef-bc21-ee6b96a634ae service placement] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:42.743750 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43629d2a-00c0-4068-b84b-97456eac264d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:42.744310 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43629d2a-00c0-4068-b84b-97456eac264d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:42.744589 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43629d2a-00c0-4068-b84b-97456eac264d admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:42.744872 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43629d2a-00c0-4068-b84b-97456eac264d admin admin] PATH_INFO: `/v3/projects/266be8998b214850a9bd63c50e8fecd9` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:42.745992 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43629d2a-00c0-4068-b84b-97456eac264d admin admin] RBAC: Authorizing `identity:delete_project(project_id=266be8998b214850a9bd63c50e8fecd9)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:42.747977 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43629d2a-00c0-4068-b84b-97456eac264d admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:42.844323 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-43629d2a-00c0-4068-b84b-97456eac264d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '266be8998b214850a9bd63c50e8fecd9', 'request_id': 'req-43629d2a-00c0-4068-b84b-97456eac264d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:42.850516 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-43629d2a-00c0-4068-b84b-97456eac264d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '266be8998b214850a9bd63c50e8fecd9', 'request_id': 'req-43629d2a-00c0-4068-b84b-97456eac264d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:42.856007 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-43629d2a-00c0-4068-b84b-97456eac264d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '266be8998b214850a9bd63c50e8fecd9', 'request_id': 'req-43629d2a-00c0-4068-b84b-97456eac264d'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:42.922820 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-43629d2a-00c0-4068-b84b-97456eac264d admin admin] The token cache is being invalidate because project 266be8998b214850a9bd63c50e8fecd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:10:42.923457 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-43629d2a-00c0-4068-b84b-97456eac264d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:42.923741 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-43629d2a-00c0-4068-b84b-97456eac264d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:42.925747 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1445/2917] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:10:42 2022] DELETE /identity/v3/projects/266be8998b214850a9bd63c50e8fecd9 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:10:42.930626 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85d457-ae1d-4237-a83e-dd4c0bf92f9e None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:42.930912 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85d457-ae1d-4237-a83e-dd4c0bf92f9e None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:42.931213 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85d457-ae1d-4237-a83e-dd4c0bf92f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:42.951377 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-dc85d457-ae1d-4237-a83e-dd4c0bf92f9e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:10:42.952588 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-dc85d457-ae1d-4237-a83e-dd4c0bf92f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:43.010871 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1473/2918] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:10:43.018375 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd50c49-147b-4c5c-a811-68f480c5cfd5 None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:43.019186 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd50c49-147b-4c5c-a811-68f480c5cfd5 None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:43.019512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd50c49-147b-4c5c-a811-68f480c5cfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:43.040437 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-acd50c49-147b-4c5c-a811-68f480c5cfd5 None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:10:43.041630 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-acd50c49-147b-4c5c-a811-68f480c5cfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:43.049489 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1446/2919] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:10:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:10:43.053816 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85d457-ae1d-4237-a83e-dd4c0bf92f9e None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:43.063506 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85d457-ae1d-4237-a83e-dd4c0bf92f9e None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:43.068056 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee14c63-77fe-4053-a57c-861a24f75f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:43.068644 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee14c63-77fe-4053-a57c-861a24f75f40 admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:43.068923 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee14c63-77fe-4053-a57c-861a24f75f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:43.069193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee14c63-77fe-4053-a57c-861a24f75f40 admin admin] PATH_INFO: `/v3/domains` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:43.069862 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee14c63-77fe-4053-a57c-861a24f75f40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:10:43.070161 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee14c63-77fe-4053-a57c-861a24f75f40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:43.071789 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee14c63-77fe-4053-a57c-861a24f75f40 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:43.073221 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:10:43.084073 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1474/2920] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Aug 30 15:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:10:43.088412 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd50c49-147b-4c5c-a811-68f480c5cfd5 None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:43.098115 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd50c49-147b-4c5c-a811-68f480c5cfd5 None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:43.102829 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8eaec1-db1c-4e7f-95bc-c9cf47ea283c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:43.103440 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8eaec1-db1c-4e7f-95bc-c9cf47ea283c admin admin] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:43.103717 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8eaec1-db1c-4e7f-95bc-c9cf47ea283c admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:43.103987 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8eaec1-db1c-4e7f-95bc-c9cf47ea283c admin admin] PATH_INFO: `/v3/projects` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:43.104619 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8eaec1-db1c-4e7f-95bc-c9cf47ea283c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:43.106508 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8eaec1-db1c-4e7f-95bc-c9cf47ea283c admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:43.110903 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:10:43.193428 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1447/2921] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Aug 30 15:10:43 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:10:43.197595 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee14c63-77fe-4053-a57c-861a24f75f40 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:43.207424 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee14c63-77fe-4053-a57c-861a24f75f40 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:43.263210 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4cf6a-2742-4d1f-b6e0-84bacb184269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:43.263743 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4cf6a-2742-4d1f-b6e0-84bacb184269 admin admin] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:43.264020 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4cf6a-2742-4d1f-b6e0-84bacb184269 admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:43.264292 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4cf6a-2742-4d1f-b6e0-84bacb184269 admin admin] PATH_INFO: `/v3/users` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:43.264911 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4cf6a-2742-4d1f-b6e0-84bacb184269 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:43.266820 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4cf6a-2742-4d1f-b6e0-84bacb184269 admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:43.374708 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1475/2922] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Aug 30 15:10:43 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:10:43.378775 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8eaec1-db1c-4e7f-95bc-c9cf47ea283c admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:43.389448 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8eaec1-db1c-4e7f-95bc-c9cf47ea283c admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:43.394061 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d2304-995f-458a-a6e7-a0568abc7ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:43.394651 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d2304-995f-458a-a6e7-a0568abc7ecd admin admin] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:43.394928 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d2304-995f-458a-a6e7-a0568abc7ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:43.395226 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d2304-995f-458a-a6e7-a0568abc7ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:43.395869 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d2304-995f-458a-a6e7-a0568abc7ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:43.397462 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d2304-995f-458a-a6e7-a0568abc7ecd admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:43.407281 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1448/2923] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:10:43.411528 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4cf6a-2742-4d1f-b6e0-84bacb184269 admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:43.421464 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4cf6a-2742-4d1f-b6e0-84bacb184269 admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:43.426937 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15657a8f-2df1-4566-bc1d-dcf959c157ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:43.427559 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15657a8f-2df1-4566-bc1d-dcf959c157ca admin admin] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:43.427866 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15657a8f-2df1-4566-bc1d-dcf959c157ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:43.428193 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15657a8f-2df1-4566-bc1d-dcf959c157ca admin admin] PATH_INFO: `/v3/roles` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:43.428842 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15657a8f-2df1-4566-bc1d-dcf959c157ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:43.430433 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15657a8f-2df1-4566-bc1d-dcf959c157ca admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:43.440527 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1476/2924] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Aug 30 15:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:10:43.444661 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d2304-995f-458a-a6e7-a0568abc7ecd admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:43.454564 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d2304-995f-458a-a6e7-a0568abc7ecd admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:43.459130 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe90cfb-67af-40b7-b9ec-f2ebba14c096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:43.459766 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe90cfb-67af-40b7-b9ec-f2ebba14c096 admin admin] REQUEST_METHOD: `PUT` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:43.460046 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe90cfb-67af-40b7-b9ec-f2ebba14c096 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:43.461107 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe90cfb-67af-40b7-b9ec-f2ebba14c096 admin admin] PATH_INFO: `/v3/projects/8319f5840f3e46cea15cd526cb93c225/users/d302c94fda6343d0a1a71599356d6ae7/roles/7c4463788f534454b6d8319dbc4d2afc` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:43.475269 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe90cfb-67af-40b7-b9ec-f2ebba14c096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8319f5840f3e46cea15cd526cb93c225, user_id=d302c94fda6343d0a1a71599356d6ae7, role_id=7c4463788f534454b6d8319dbc4d2afc)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:43.478007 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe90cfb-67af-40b7-b9ec-f2ebba14c096 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:43.580575 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1449/2925] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Aug 30 15:10:43 2022] PUT /identity/v3/projects/8319f5840f3e46cea15cd526cb93c225/users/d302c94fda6343d0a1a71599356d6ae7/roles/7c4463788f534454b6d8319dbc4d2afc => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:10:43.591433 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02d72d-d9ab-43d4-992c-bffdd2b1f500 None None] REQUEST_METHOD: `POST` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:43.591716 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02d72d-d9ab-43d4-992c-bffdd2b1f500 None None] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:43.591994 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02d72d-d9ab-43d4-992c-bffdd2b1f500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:43.619788 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.auth.core [None req-3e02d72d-d9ab-43d4-992c-bffdd2b1f500 None None] MFA Rules not processed for user `d302c94fda6343d0a1a71599356d6ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=88058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:10:43.620998 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-3e02d72d-d9ab-43d4-992c-bffdd2b1f500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:43.684638 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1477/2926] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:10:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:10:43.696295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe90cfb-67af-40b7-b9ec-f2ebba14c096 admin admin] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:43.697986 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-6fe90cfb-67af-40b7-b9ec-f2ebba14c096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:43.753675 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe90cfb-67af-40b7-b9ec-f2ebba14c096 admin admin] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:43.757430 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1bb64-99dd-4b6f-8548-db4fbd2658b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef389656cc8546fdb7d9f30329b93dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:43.758028 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1bb64-99dd-4b6f-8548-db4fbd2658b4 service cinder] REQUEST_METHOD: `GET` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:43.758309 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1bb64-99dd-4b6f-8548-db4fbd2658b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:43.758582 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1bb64-99dd-4b6f-8548-db4fbd2658b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:43.766743 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1bb64-99dd-4b6f-8548-db4fbd2658b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:43.768212 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1bb64-99dd-4b6f-8548-db4fbd2658b4 service cinder] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:43.777187 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1450/2927] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:10:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:10:46.840295 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02d72d-d9ab-43d4-992c-bffdd2b1f500 None None] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:46.904444 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02d72d-d9ab-43d4-992c-bffdd2b1f500 None None] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:46.908913 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d135c-4d51-4b5b-9cad-0e91bed5dd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:46.909539 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d135c-4d51-4b5b-9cad-0e91bed5dd1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:46.909814 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d135c-4d51-4b5b-9cad-0e91bed5dd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:46.910087 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d135c-4d51-4b5b-9cad-0e91bed5dd1f admin admin] PATH_INFO: `/v3/users/d302c94fda6343d0a1a71599356d6ae7` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:46.911289 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d135c-4d51-4b5b-9cad-0e91bed5dd1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d302c94fda6343d0a1a71599356d6ae7)` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:46.913208 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d135c-4d51-4b5b-9cad-0e91bed5dd1f admin admin] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:46.915551 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:10:47.153413 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-380d135c-4d51-4b5b-9cad-0e91bed5dd1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd302c94fda6343d0a1a71599356d6ae7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:47.160115 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-380d135c-4d51-4b5b-9cad-0e91bed5dd1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd302c94fda6343d0a1a71599356d6ae7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:47.172146 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-380d135c-4d51-4b5b-9cad-0e91bed5dd1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd302c94fda6343d0a1a71599356d6ae7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:47.172803 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-380d135c-4d51-4b5b-9cad-0e91bed5dd1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd302c94fda6343d0a1a71599356d6ae7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:47.173407 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.notifications [None req-380d135c-4d51-4b5b-9cad-0e91bed5dd1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd302c94fda6343d0a1a71599356d6ae7'} {{(pid=88058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:47.266009 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1478/2928] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Aug 30 15:10:46 2022] DELETE /identity/v3/users/d302c94fda6343d0a1a71599356d6ae7 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:10:47.270658 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcbc60-52b9-4104-abd4-5eeff2206f5e None None] REQUEST_METHOD: `POST` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:47.271066 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcbc60-52b9-4104-abd4-5eeff2206f5e None None] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:47.271253 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcbc60-52b9-4104-abd4-5eeff2206f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:47.290512 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.auth.core [None req-59dcbc60-52b9-4104-abd4-5eeff2206f5e None None] MFA Rules not processed for user `ef040204e84943a7ad9a173f7dd9bb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=88059) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:10:47.291709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-59dcbc60-52b9-4104-abd4-5eeff2206f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:47.349418 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1451/2929] 10.222.0.27 () {58 vars in 958 bytes} [Tue Aug 30 15:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:10:47.359123 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d135c-4d51-4b5b-9cad-0e91bed5dd1f admin admin] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:47.360573 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-380d135c-4d51-4b5b-9cad-0e91bed5dd1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:47.418647 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d135c-4d51-4b5b-9cad-0e91bed5dd1f admin admin] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:47.422520 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120e46f-30be-4111-8c18-a26edede4e5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bff0664e0b44b7c85344a9f3e88f126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:47.423142 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120e46f-30be-4111-8c18-a26edede4e5a service neutron] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:47.423425 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120e46f-30be-4111-8c18-a26edede4e5a service neutron] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:47.423711 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120e46f-30be-4111-8c18-a26edede4e5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:47.432817 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120e46f-30be-4111-8c18-a26edede4e5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:47.434225 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120e46f-30be-4111-8c18-a26edede4e5a service neutron] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:47.443599 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1479/2930] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:10:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:10:47.843191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcbc60-52b9-4104-abd4-5eeff2206f5e None None] Authenticating user token {{(pid=88059) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:47.843191 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.fernet_utils [None req-59dcbc60-52b9-4104-abd4-5eeff2206f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88059) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:47.852958 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcbc60-52b9-4104-abd4-5eeff2206f5e None None] Validating token access rules against request {{(pid=88059) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:47.859058 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48c478-e49c-42ef-b615-874b2332dcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef040204e84943a7ad9a173f7dd9bb2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd707a58a86f74ccbb5ffb499d9409139', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88059) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:47.859768 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48c478-e49c-42ef-b615-874b2332dcd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:47.860038 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48c478-e49c-42ef-b615-874b2332dcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:47.860242 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48c478-e49c-42ef-b615-874b2332dcd5 admin admin] PATH_INFO: `/v3/projects/8319f5840f3e46cea15cd526cb93c225` {{(pid=88059) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:47.861571 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48c478-e49c-42ef-b615-874b2332dcd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8319f5840f3e46cea15cd526cb93c225)` {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:47.863638 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48c478-e49c-42ef-b615-874b2332dcd5 admin admin] RBAC: Authorization granted {{(pid=88059) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:47.991320 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:10:47.992709 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5f48c478-e49c-42ef-b615-874b2332dcd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8319f5840f3e46cea15cd526cb93c225', 'request_id': 'req-5f48c478-e49c-42ef-b615-874b2332dcd5'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:47.998794 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5f48c478-e49c-42ef-b615-874b2332dcd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8319f5840f3e46cea15cd526cb93c225', 'request_id': 'req-5f48c478-e49c-42ef-b615-874b2332dcd5'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:48.004893 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5f48c478-e49c-42ef-b615-874b2332dcd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8319f5840f3e46cea15cd526cb93c225', 'request_id': 'req-5f48c478-e49c-42ef-b615-874b2332dcd5'} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:48.099295 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5f48c478-e49c-42ef-b615-874b2332dcd5 admin admin] The token cache is being invalidate because project 8319f5840f3e46cea15cd526cb93c225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88059) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:10:48.099587 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5f48c478-e49c-42ef-b615-874b2332dcd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:48.100324 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG keystone.notifications [None req-5f48c478-e49c-42ef-b615-874b2332dcd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88059) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:10:48.101971 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: [pid: 88059|app: 0|req: 1452/2931] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Aug 30 15:10:47 2022] DELETE /identity/v3/projects/8319f5840f3e46cea15cd526cb93c225 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:10:57.439437 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:10:58.197902 dv-cinder-828807-5-iscsi devstack@keystone.service[88059]: DEBUG dbcounter [-] [88059] Writing DB stats keystone:SELECT=7,keystone:DELETE=3,keystone:INSERT=1 {{(pid=88059) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:14:55.897554 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120e46f-30be-4111-8c18-a26edede4e5a service neutron] Authenticating user token {{(pid=88058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:14:55.899159 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-5120e46f-30be-4111-8c18-a26edede4e5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:14:56.041427 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120e46f-30be-4111-8c18-a26edede4e5a service neutron] Validating token access rules against request {{(pid=88058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:14:56.100849 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f24be6-1e5c-4313-8d30-08a169f434fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4f11655cd51438e9652869f74ffa975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1147574d59c7404baf76a24149684f6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:14:56.101378 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f24be6-1e5c-4313-8d30-08a169f434fd service placement] REQUEST_METHOD: `GET` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:14:56.101663 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f24be6-1e5c-4313-8d30-08a169f434fd service placement] SCRIPT_NAME: `/identity` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:14:56.101929 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f24be6-1e5c-4313-8d30-08a169f434fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:14:56.103752 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.fernet_utils [None req-67f24be6-1e5c-4313-8d30-08a169f434fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:14:56.115606 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f24be6-1e5c-4313-8d30-08a169f434fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:14:56.117221 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f24be6-1e5c-4313-8d30-08a169f434fd service placement] RBAC: Authorization granted {{(pid=88058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:14:56.127073 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: [pid: 88058|app: 0|req: 1480/2932] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Aug 30 15:14:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:15:06.122805 dv-cinder-828807-5-iscsi devstack@keystone.service[88058]: DEBUG dbcounter [-] [88058] Writing DB stats keystone:SELECT=89 {{(pid=88058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}